Misplaced Pages

:Bots/Requests for approval/ProtectionBot - Misplaced Pages

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
< Misplaced Pages:Bots | Requests for approval

This is an old revision of this page, as edited by Carcharoth (talk | contribs) at 10:24, 7 January 2007 (→Discussion: comment). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Revision as of 10:24, 7 January 2007 by Carcharoth (talk | contribs) (→Discussion: comment)(diff) ← Previous revision | Latest revision (diff) | Newer revision → (diff)

ProtectionBot

Operator: Dragons flight

Automatic or Manually Assisted: Automatic

Programming Language(s): Python (partially dependant on pywikipedia, but mostly original code)

Function Summary: Intended to automatically protect templates and images appearing on the main page or today's featured article, and unprotect them when they are no longer in these high profile locations. This bot will require sysop rights.

Edit period(s) (e.g. Continuous, daily, one time run): Every 15 minutes

Edit rate requested: N/A

Already has a bot flag (Y/N): N

Function Details: The bot reads Main Page and Main Page/Tomorrow, identifies the current and immediate future featured articles, and then generates a list of all templates/images appearing on any of these four pages. The bot then checks each template and image to see whether it is protected (i.e. edit=sysop, move=sysop). If it is not protected, the bot will automatically protect the template/image, and add that item to an internal list. The bot will also check this internal list for items it has protected in the past but which are no longer in high profile locations and restore those items to their pre-existing state. It will not remove protection that had been added by someone else.

I have already written code capable of performing most (but not all) of the above functions, but I am coming here to get feedback from the community before carrying this project further. As noted above, this bot would require sysop rights in addition to ordinary bot approval. For an alternative but less extensive proposal, see: Misplaced Pages:Bots/Requests_for_approval/Shadowbot2. Dragons flight 11:32, 30 December 2006 (UTC)

Addendum: In order to protect images which come from Commons, I have added an ability to upload them to Misplaced Pages. When the local copy is no longer needed, I would have the bot tag them for speedy deletion. Dragons flight 06:37, 1 January 2007 (UTC)

To expand on this, I am proposing that the bot copy the image, image description, and any neccesary templates that exist on Commons but not locally. When the local copy was no longer needed, the image would be tagged for deletion. Any commons image templates would be left here since presumably they could be used for future images. Dragons flight 01:09, 3 January 2007 (UTC)

Addendum 2: Per suggestion, it would also track images/templates used within Template:Did you know/Next update. Dragons flight 10:46, 1 January 2007 (UTC)

Update: I now have an essentially fully functional code base, and have updated User:ProtectionBot to give more operational details. Dragons flight 00:43, 2 January 2007 (UTC)

Discussion

  • support, we need this. Giving it admin status is no problem as long as it is assured that it will only ever use them for the task described, and as long as it has a giant "nuke me now" button in case it should malfunction. dab (𒁳) 11:59, 30 December 2006 (UTC)
  • Support --Ligulem 12:52, 30 December 2006 (UTC)
  • Support - On AN you also mentioned protecting templates and images for the article of the day, but this is not specified above. That should be clarified. (Misread section) I'm fine with the proposal so long as the article of the day page itself is not protected... the blurb about it on the main page and all templates/images displayed on it can be protected, but the article itself should be open to editing. Note that this might result in a vandal image being added to the page and then protected by this bot... but someone could then still revert to remove the image from the page itself. Also note that Main Page/Tomorrow duplicates all but four of the templates used on the Main Page and thus it might be better to check some of those templates directly. Specifically, the other templates are the 'article of the day', 'picture of the day', and 'on this day' sections for the following day and Template:Did you know/Next update in place of Template:Did you know. The 'next update' page often gets updated several times per day and is currently accessible to non-admins and thus might not be appropriate to protect prior to be placed on the Main page. --CBD 13:21, 30 December 2006 (UTC)
  • Comment - every 15 minutes is not good enough. That would give a vandal or an unauthorised (well, obviously it would be unauthorised!) vandal-bot 15 minutes to act first. Suggest changing to a lower but unspecified amount of time so that vandals are not aware of how long they have. Ideally, if possible, make it impossible to add unprotected images and templates to these sensitive areas. Can we also come up with a bot-assisted plan to deal with the template vandalism that was seen on 'today's featured article' for about a week, as that will surely return at some point unless it is dealt with now. Carcharoth 15:04, 30 December 2006 (UTC)
    As described it would be looking at Main Page/Tomorrow and Tomorrow's Featured Article as well as the current ones, so predictable elements will be protected before they actually reach high profile status. The unpredictable elements (such as a news event causing a new image to appear on In The News) would be caught the next time the bot runs, but ideally should still be protected by the admin who added them. It has been the predictable elements that have been greatest sources of vandalism lately, and so that ought to be eliminated. I also plan to add random jitter to the bot execution times, so as to make it more difficult for anyone to race the bot. Dragons flight 18:28, 30 December 2006 (UTC)
    Also, per the comment here, what are the mechanisms in place for alerting people as to whether the bot is working or not working. If the bot is approved and successful, people will gradually let the issue drift to the back of their minds and come to rely on the bot. How can we ensure that there is always someone watching the bot (more than one person and not just the person running the bot)? Carcharoth 15:09, 30 December 2006 (UTC)
    Carcharoth raises a very crucial issue here. Shadowbot2 has also had a problem as well, see User_talk:Shadowbot2#Problem. Because it sends an email, we were able to detect it quickly. Zzyzx11 (Talk) 15:53, 30 December 2006 (UTC)
    If it is expected to be offline, I would warn people. At present I have no other plans for ways to make sure any unplanned downtime is noticed, but I am open to suggestions. Dragons flight 18:28, 30 December 2006 (UTC)
    I've added the function described here to provide a limited sort of warning if it stops working. Dragons flight 04:14, 2 January 2007 (UTC)
  • About time for this to come here, I have full support of this bot and bot operator, you have done great work at WP:RFASUM and as long as you are willing to share the code with other admins via e-mail or other secure method I have no problem with this bot not being approved. Betacommand 15:22, 30 December 2006 (UTC)
  • Question: Since the plan is for the bot to also protect images that appear on the main page, how will it deal with those from Commons? The current procedure is to upload a local copy here onto Misplaced Pages, tag it with {{C-uploaded}}, and protect that. Protecting just the image description page here does not work because a vandal can simply log onto Commons using a sleeper account and save a new version of the image there. Zzyzx11 (Talk) 16:05, 30 December 2006 (UTC)
    • The code written so far would ignore images uploaded only to Commons, but ultimately, the ideal behavior is probably to do what admins currently do and upload a copy locally for protection. This will take a bit of time to implement, but certainly seems doable. After it was no longer needed, I could have to bot tag the image for local deletion (I prefer doing that than having the bot actually delete the image, as an extra safeguard against rogue deletion behavior). Dragons flight 18:28, 30 December 2006 (UTC)
      • I'm not sure I quite understand what you mean by " would ignore images uploaded to Commons" - does that mean they would remain unprotected unless a (human) admin uploads and then protects the images? This would be a serious concern. Flcelloguy (A note?) 23:18, 30 December 2006 (UTC)
        • I now have code prepared to automatically copy images over from Commons so that they can be protected locally. Dragons flight 06:37, 1 January 2007 (UTC)
          • Thanks for clarifying this! A few more questions/comments: would the bot be programmed to delete the image once it is off the Main Page? If there are prior revisions in the image description here (i.e. FA tagging) will the bot restore those revisions? In addition, does the bot also copy the image information (i.e. description and licensing information) from Commons? If it does, will it recognize when a template used on Commons isn't used here, leading to a red link? (In other words, Commons had a template that we didn't have; I've had to manually substitute the code on Commons, sometimes several times for templates within templates, copy the code here, and then revert myself on Commons.) Thanks! Flcelloguy (A note?) 03:54, 2 January 2007 (UTC)
            • When the local image is no longer needed, it is programmed to add a tag that marks the image for speedy deletion and includes a reminder to the deleting admin to restore any pre-existing description page content after deleting the local copy. Given the unusual nature of this project, I felt it was prudent not to go handing direct deletion authority to a bot. It does not copy the description from Commons but rather provides a template that points at the Commons description page, similar to {{c-uploaded}}. Dragons flight 04:10, 2 January 2007 (UTC)
              • There might be a problem if you do not copy the description from Commons. See this edit summary. Zzyzx11 (Talk) 17:29, 2 January 2007 (UTC)
                • Sigh. Have I complained recently about how the existence of Commons makes life here more difficult? Okay, I've modified the bot code to be able to copy over the Commons image descriptions (including duplicating any unique to Commons image tags). Dragons flight 01:09, 3 January 2007 (UTC)
                  • Maybe leaving away all bot-features relating to images for a start? The images thing starts getting very complicated (just and idea, I'm not opposing anything). Concentrating on the templates first wouldn't be that bad. --Ligulem 11:50, 3 January 2007 (UTC)
  • Comment - I personally think that the templates and images on the main page need to be watched by more sets of human eyes. Historically, most of the disruption on the main page is caused by sysops making small changes without consensus and often introducing bad markup in the process. Every template and image on the main page should be checked by human eyes after any edit is made. I fear that adding a bot will reduce the amount of redundant checks on these pages and thus cause minor mistakes to stay up even longer. --- RockMFR 19:21, 30 December 2006 (UTC)
    • I have to agree with the general sentiment of this comment; I'm always wary of entrusting a task to a bot without continuous human oversight. As pointed out above, there is no mechanism to ensure that admins are alerted if the bot is malfunctioning or stops working, and the results could prove disastrous. The current system works, and given the new system of checks with Shadowbot2, I don't feel that there is an urgent need for the bot, especially with the (very real and plausible) chances of malfunction or stoppage. Thanks! Flcelloguy (A note?) 23:24, 30 December 2006 (UTC)
  • Comment: If by some miracle the community emerges from its haze of ignorance and and actually permits this bot to acquire a sysop flag, then for god's sake use the precedent to get the same thing done with other bots that need it. There's a bloody user javascript subpage transcluded on to the Main Page because people refuse to give the FA-counting bot the necessary flag. In other words, they're happy to let it edit a template on the Main Page via some clumsy hack, but they won't sysop it so it can do so the normal way – Gurch 20:23, 30 December 2006 (UTC)
    • I think the concerns raised by people over an admin-bot were (and continue to be) valid, even though I don't necessarily agree with all of them; I also hope that following discussion here, any attempts to make the bot a sysop will go through the usual channel of RfA, which seems to be more appropriate (i.e. it's the place for all requests for admin rights, and would have a wider participation than here, on the bot request page.) Thanks! Flcelloguy (A note?) 23:29, 30 December 2006 (UTC)
  • Support though note that usually the admin protecting a page looks at the page before protecting it; if this bot protects an already vandalized template it will be harder for regular users to undo that vandalism. — brighterorange (talk) 23:35, 30 December 2006 (UTC)
    • For that matter, the bot would also not be able to recognize vandalized images prior to protection. Although (hopefully) people will catch the vandalism prior to protection or, at latest, prior to the image being shown on the Main Page, this contributes to my queasiness about eliminating human oversight. Thanks! Flcelloguy (A note?) 23:48, 30 December 2006 (UTC)
      • I agree. If the bot cannot recognize vandalized images and templates, then I would rather have non-admins constantly revert those files until another sysop can step in. Zzyzx11 (Talk) 00:28, 31 December 2006 (UTC)
      • No, the bot couldn't tell a vandalized image/template from a non-vandalized one... but should it need to? Think about the way it works. It would protect all 'daily' content on the main page (except 'In the news' and 'Did you know') starting at midnight the day BEFORE it went live. So maybe a vandal could sneak something in just before midnight and the first bot protection... causing the bot to then protect the vandalized material. However, that would then be visible for an entire day before it went live. There'd still be a need for 'human eyes', but we'd have a full day to catch vandalism before it was widely distributed. After midnight of the day before 'activation' the page would be 'safe' unless an admin vandalized or unprotected it... and even the latter would only last for a few minutes. The 'article of the day' would work the same way except that since that page itself would be unprotected someone could add a vandalized image or template to it... which the bot might then protect. However, the article would still not be protected and any other user could just revert/remove the call to the vandalized image or template. There would be 'vulnerabilities' if this proposal were implemented (including some I haven't mentioned to avoid educating the vandals), but they would be significantly less than those which exist now. --CBD 10:44, 31 December 2006 (UTC)
        • I agree. Unless the vandalism is very subtle or there are no admins looking (unlikely) then this is not really a problem. I just wanted to point out that part of the act of an admin protecting a template from vandalism (which is checking that the template is not currently vandalized) can't be replicated by a bot, so we still need some oversight. — brighterorange (talk) 16:26, 31 December 2006 (UTC)
          • Though the chances of either an image or template being vandalised and then protected are slim, I've seen it happen before; vandals hit the images/templates scheduled to come up in a few days. Although rare, I've (luckily) caught this before, and although I'm also confident that (hopefully) an admin would catch this either before to or after the protection, I'm still hesitant with a bot doing this. (Note that vandals also adapt to our tactics; once they figured out what time the bot protects, they could easily do some damage if no one was watching the process.) Perhaps some checks could be implemented here, if possible? Thanks! Flcelloguy (A note?) 04:01, 2 January 2007 (UTC)


Non-admins could still remove said template from the featured article. The only case where it might be a problem is if that "newly-added template" was used widely on hundreds of other pages. If that was the case, an admin would be needed urgently to undo protection and fix things for the hundreds of other articles where the template was being used. But high-use templates should already be protected anyway. Carcharoth 10:10, 7 January 2007 (UTC)
  • Oppose Security through obscurity is not. Release the code for everyone to try to break or just have it add "needs-protection" templates to articles/images and do the commonsuploadingreplacement-tagging. If some smart vandal uploading a buffer overrun can block a user, then it cannot run with a bit. Is there anything in the code worth keeping secret? Only if there's a mistake in it. Hipocrite - «Talk» 21:45, 5 January 2007 (UTC)
  • Support. I support this more eagerly than any other proposal I have ever voted for on Misplaced Pages. RyanGerbil10(Упражнение В!) 23:52, 5 January 2007 (UTC)
  • Support. Requested something similar myself...great minds think alike!--HereToHelp 03:29, 6 January 2007 (UTC)
  • Support - we need this urgently to stop the template vandalism that hits the main page. Moreschi 16:54, 6 January 2007 (UTC)
  • Support ~Crazytales 21:46, 6 January 2007 (UTC)
  • Comment - what would happen if a template was removed from the featured article during the course of the day. Would ProtectionBot remember it, and would it still unprotect it at the end of the day, or would unprotection happen sooner, at the next interval? ie. Would all unprotection be queued until the end of the day? Carcharoth 10:10, 7 January 2007 (UTC)
  • Comment - there is the point that we still need to LOOK at the Main_Page/Tomorrow to pick up vandalism, in fact we need to do this more than ever before. If this works, people will get complacent and the inevitable result will be that a vandal slips something in just before protection takes place, the vandalism (possibly not very visible) remains undetected for a whole day, and then silently switches over on the main page, at which point all hell will break loose. Checking the 'tomorrow' version will also guard against various 'date template' vandalism possibilities. Carcharoth 10:24, 7 January 2007 (UTC)

Trials

Approved for trial. Please provide a link to the relevant contributions and/or diffs when the trial is complete. Well it looks like there is more then enough support to run trials with this bot, so Conditional Trial Approved if it is possible to run this without admin rights (perhaps generating the lists so we can verify the parsing?) As this type of test will produce beany results if it can post the results to a page that is then swiftly deleted admins can review the results. Due to the expected high visibility of this bot lets also leave this discussion transcluded on WP:RFBOT for now. — xaosflux 05:16, 5 January 2007 (UTC)

I've already run all the trials I feel are necessary short of full-scale deployment. You really want me to sit around generating lists to show that I know how to parse text? That feels very much like a waste of time to me, and potentially hazardous as well if made public. The current and tomorrow's main page/featured article collectively contain 53 templates and 31 images. All 53 templates are already protected as are 20 of the 31 images. Of the 11 unprotected images (none on the main page currently, thankfully), 6 are local and 5 reside on commons. Parsing out the relevant bits is somewhere around step 2 in a 10 step process. You are welcome to review the code, as several other people have volunteered to do, but if we are going to have a trial I would ask that it be a full demonstration that it works, rather than making extra pointless work for me. Dragons flight 06:33, 5 January 2007 (UTC)
I agree with Dragons flight the only way to test is to actually run the bot on the main page. Unless Xaosflux or any other BAG has an objection I give my support for a 24 hour trial (this will cover todays FA and tomorrow's FA protection, and the unprotection of the current FA once it leaves the main page) Betacommand 06:47, 5 January 2007 (UTC)
How many humans will be watching our robot overlords during the test? :-) Carcharoth 14:28, 5 January 2007 (UTC)
Some serious comments. I like the safety valve feature, but I wonder if there is a way to test that as well to make sure the "not editing for 90 minutes" thing does work? Also, is it possible to make clear to admins using the emergency shut-off button that they should follow-up and make sure that protection is dealt with by humans, rather then just walking away with a "bot's shut off now, everything should be OK now" shrug of the shoulders. That's the only thing I can think of at the moment. All looks really good! Carcharoth 14:38, 5 January 2007 (UTC)
The safety valve is controlled by MediaWiki itself through template syntax, so as long as it's tested once to make sure the math is right and it does kick in after 90 minutes, it should work the same every time. Also, blocking the bot would trigger said safety valve 90 minutes after the block; since the bot would theoretically have already handled the next day's protection, I think warning within 90 minutes should be plenty. —bbatsell ¿? 17:18, 5 January 2007 (UTC)
I am not suggesting you can not parse code, just that this request needs to have a signfigant degree of caution used for it, and I am all for other suggestions on how to trial this bot, my trial approval was to continue to move this request forward. The hightened caution is twofold, first it is due to this requests interaction with our most viewed page; and second is that it will end up requiring an WP:RFA and getting full support and showing all due dillegence here on RFBOT should alleviate challenges that have occured on RFA during other bot attempts. — xaosflux 04:25, 6 January 2007 (UTC)
  • In the last admin bot BRFA, it was felt that the lack of support shown on the BRFA carried through to the RFA's failure. By showing our support here we are avoiding this ill fate and showing the BAG that there is community (or cabal Template:Emot) approval of the plans. In any case, anyone can express their opinion in a BRFA, or their support for a proposal - in fact I went and commented on a few today. Helpful suggestions or just plain support for an idea can help :) Martinp23 20:43, 5 January 2007 (UTC)

Comments from Trial

I have begun running this on my admin account on a manual basis. In doing so I have identified several issue:

  1. A unicode handling bug in one part of the program that crashed it. This is now fixed.
  2. A problem with dropped wikisession information. I suspect that this was caused by how Mediawiki handles sessions and a conflict with my trying to use my account at the same time the bot was running. If I am correct, this means that this bot cannot run on my account at the same time that I am using it, and consequently continuous operation absolutely requires a seperate account. I will be investigating this further.
  3. A formatting issue wherein the bot converted a newline to a space. I haven't yet tracked down why this occured.
    The formatting issue should be resolved now (albeit with an ugly hack). Dragons flight 04:17, 6 January 2007 (UTC)

The first two errors, being fatal, required me to do some substantial manual cleanup after the bot during this initial run. It has also suggested ways that I can improve the error handling which I intend to do now. Aside from these issues, it appears to be working as designed. Dragons flight 01:14, 6 January 2007 (UTC)

Thanks for the trial report. A comment: would it be possible for the bot to add the images it protects to Category:Protected main page images? I'm not sure if the bot is actually copying the code from the template pages or taking the code from the source (I haven't gotten a chance to go through the code you sent me ;-) ), so all I've done is place the templates themselves in the category. Or would a creation of a new category for images not on the Main Page but on the FA itself be more appropriate? Thanks! Flcelloguy (A note?) 04:14, 6 January 2007 (UTC)
I was thinking that "Protected by ProtectionBot" and "Uploaded by ProtectionBot" could be made subcats of all protected/uploaded high profiles stuff. Dragons flight 04:22, 6 January 2007 (UTC)

(edit conflict) I have also discovered that the list of transcluded templates does not respond to action=purge and can at times be several hours out of date. Consequently I implemented a work around for this that forces the template list to update. Dragons flight 04:17, 6 January 2007 (UTC)

I will be leaving the bot running while I go away for a bit. Since this is the first time it has been left running without myself directly in front of it, I thought I should let others know. It has been operating normally for quite a while now and I don't anticipate any problems, especially since there is little activity on the Main Page this time of day. Dragons flight 07:21, 6 January 2007 (UTC)

Today's midnight UTC changeover seems to have gone as expected. The bot failed to unprotect a couple images, but this was caused by missing log entries related to yesterday's bugs, and not any current problem in the bot. During the night it also served to protect and unprotect some changes that where made to the Did you know image. Dragons flight 01:13, 7 January 2007 (UTC)

Thanks for the updates! Two minor requests: first, could you have your bot unprotect old selected anniveraries as well? The "tradition", per se, has always been to unprotect them when they are off the Main Page so that they can be updated accordingly (new events will be added, holidays that don't stay on the same day will need to be changed, etc.). I had to just manually unprotect the SA for January 6. Secondly, would you mind adding "BOT:...", like you have done with normal edits, to the protection log summary while the bot is running off of your account? This would make it a lot easier to look at the contributions, etc. Thanks a lot! Flcelloguy (A note?) 02:11, 7 January 2007 (UTC)
Yeah, I've noticed that some of those selected anniversaries have been protected a really long time as well. My thought on that had been that I would propose a seperate one-time run to clear all the really old protections some time after ProtectionBot was fully set up and deployed. On your second point, I can add "Bot:", but frankly it's already more than a little inconvenient as a developer/operator to have this running on my own account. I would much prefer to fully set up the seperate account as soon as possible, so that the bot's actions would be cleanly seperated from my own. See for reference this discussion and the proposal at the end. Even a temporary sysop flag would make it easier for me and others to see what it is doing. Dragons flight 03:12, 7 January 2007 (UTC)
As a bot running it probably should have had those words on the edit summery. Cheers! —— Eagle 101 05:29, 7 January 2007 (UTC)

End of Trials

Please note that this bot will only be approved of if WP:OFFICE steps in or RfA takes place. Dragons flight I suggest that you post the RfA as soon as possible so that we can get the flag for this bot.In regard of the approved test that should have concluded by now, if more testing needs to be done before the RfA is over please test on as the trial was just given to work out any kinks before the RfA. If this bot has as much support as the BRfA has the closing B-crat can take into consideration all of the anti-adminbot biasis when they close it since it is a discussion and not a election. Betacommand 05:04, 7 January 2007 (UTC)

As requested, I've shut it down and filed the RFA. I have posted a notice of this at AN, that also includes a request that it be allowed to continue running during the RFA. Dragons flight 09:54, 7 January 2007 (UTC)
Category: