Misplaced Pages

Revision as of 17:54, 18 March 2011

Snotbot 4

• (BRFA
• Approved BRFAs
• talk
• contribs
• count
• SUL
• logs
• page moves
• block log
• rights log
• flag)

Operator: Snottywong (talk · contribs)

Time filed: 23:24, Monday March 14, 2011 (UTC)

Automatic or Manually assisted: Automatic

Programming language(s): Python

Source code available: Pywikipedia

Function overview: Fix duplicate references.

Links to relevant discussions (where appropriate): Per Misplaced Pages:CITE#Footnotes. Let me know if centralized discussion is necessary for this task.

Edit period(s): Will run once to clear out the current backlog, then run intermittently thereafter if the backlog becomes large again.

Estimated number of pages affected: Current backlog is 5,589 per toolserver.

Exclusion compliant (Y/N): No

Already has a bot flag (Y/N): Yes

Function details: The bot will work off the list of articles provided at the reference duplication toolserver script. Specifically, it will check each article for duplicate references. If it finds multiple copies of the text:

<ref>foobar.com</ref>

it will replace the first instance with:

<ref name=duplicateref1>foobar.com</ref>

and it will replace all subsequent instances with:

<ref name=duplicateref1 />

Unless someone has a better idea for a ref naming scheme.

Discussion

• In general, it is a good idea. The only issue I can think of is that, not very often, I'll use duplicate references when I want to use two different URL links to two different pages in the same book posted at Google. If the bot were to remove one of my two URLs, that would be a mistake. -- Uzma Gamal (talk) 10:04, 15 March 2011 (UTC)
• I assume this deals with exact same content between refs (+- whitespace)? —  HELLKNOWZ  ▎TALK 10:06, 15 March 2011 (UTC)
• Needs wider discussion. The use of named references is opposed by some editors, see Misplaced Pages:Village pump (policy)/Archive 74#Repeating a reference but doing something different for one example (there are links to other examples in there too). Anomie⚔ 11:27, 15 March 2011 (UTC)

To be clear, the bot is only intended to find references that match exactly. If there are minor differences between the <ref> tags, the bot will not touch it. Even if there are two identical refs which use a {{cite}} template, but the arguments in the template are in a different order, the bot won't touch them. All it's going to do is look for <ref></ref> tags, take note of what's between them, and look for exact matches elsewhere in the article. So, in both Uzma Gamal's case and the cases brought up at the VP thread, those potential issues will not be a problem. I'll start a discussion to be sure though. —SW—  14:23, 15 March 2011 (UTC)

Discussion started at Misplaced Pages:Village pump (proposals)#Bot to reduce duplicate references. —SW—  16:57, 15 March 2011 (UTC)

• Very strongly Oppose. Whether we should have named references or not (and correspondingly, whether we should have multiple footnotes at a single point) are matters of editorial judgment; an article repeating one reference exactly is not a problem - and will avoid other problems. The examples given in the discussion show clearly that the creator envisages only articles using web sources (for which the system of named footnotes is usually appropriate); but we have many articles which are not. Septentrionalis PMAnderson 17:21, 16 March 2011 (UTC)

Naming references is an editorial decision, and not one that I am trying to interfere with. There is no logical editorial process that would lead to the decision to have multiple references in an article that are 100% identical. There are plenty of reasons why you'd want to have multiple references to a single source, where each reference is slightly different (i.e. different page numbers, chapters, comments, quotes, etc.), and this bot will not affect those articles at all. In the unlikely event that duplicate references were created as a result of a conscious editorial decision, then it was the result of a bad editorial decision which should be corrected. Furthermore, the table that I posted at the village pump discussion clearly shows that the intention is not to only affect articles using web sources. Several of the examples use the {{cite book}} template, which is clearly not for web sources. The bot will not differentiate between varying types of sources, it will only look for identical wikitext between the <ref> tags. I'm interested to learn about specific example cases where you believe the operation of this bot will cause problems. —SW—  17:56, 16 March 2011 (UTC)

Absolutely oppose. Of course there is a logical process which would lead to absolutely identical footnotes: citing precisely the same source at different points in the article. Print sources decrease this slightly with ibid. and loc. cit., but there is consensus that this is too dangerous for us, since any rearrangement may make these into errors - and they repeat ibid. anyway. Septentrionalis PMAnderson 02:23, 18 March 2011 (UTC)

Please get a clue, if you are going to cite the same ref you should not be using ibid or anything like that, what you should be doing is creating a reference and then using <ref name="refname"/> to refer to it. Anything else will lead to a fuck up. If you're using ibid and someone else adds an additional reference between your original source and the use of ibid ibid now refers to the wrong ref. The person who decides to add a single ref should not have to worry about not breaking all the refs on a given page. ΔT 02:29, 18 March 2011 (UTC)

Please read WP:FOOTNOTE. Named footnotes are one solution; they are not mandatory; depending on the article, they may be distinct disimprovements. If this bot is equally badly written in other respects, it should be stopped summarily. Septentrionalis PMAnderson 02:34, 18 March 2011 (UTC)

Let me quote that back to you since you cannot seem to bother reading the full thing

Not sure you can get much clearer than that. thus your argument is dead. ΔT 02:41, 18 March 2011 (UTC)

There is now an expansive thread on Misplaced Pages:Village pump (proposals)#Bot to reduce duplicate references with very broad support and a small but vocal minority of opposition (consisting of 2 editors). If you read through the entire thread, you'll see that none of your arguments are even remotely persuasive. You can say all day long that "named footnotes may be distinct disimprovements" and that "sequentially numbered footnotes are sometimes preferable", but until you actually provide evidence that any of these things are true, your argument falls on deaf ears. "Evidence" doesn't mean "your opinion", nor does it mean pointing to irrelevant external style guides. I've said it too many times to count, but I'll say it one more time because it has not yet been refuted: There is no logical reason to ever have duplicate references that have not been grouped by naming them. There's just not. Take a look at every FA on the site, I guarantee you won't find one example. If you do find an example and point it out, it would be quickly fixed. If you submitted an article to WP:FAR with duplicate references, it would not pass until they were fixed. For these and so many other reasons, I still believe this bot should run its task. I believe the thread on the village pump shows consensus to continue, and I hope someone from BAG will allow a trial to commence soon. Also note that Pmanderson apparently has some kind of problem with me and/or my bot (despite never having run into him before to my knowledge), as he has taken it upon himself to oppose every bot request I have open at the moment. —SW—  03:09, 18 March 2011 (UTC)

See also Help:Footnotes#Multiple_citations_of_the_same_reference_or_footnote, which does not mention that this is optional or subject to stylistic interpretation in any way. Case closed. —SW—  03:46, 18 March 2011 (UTC)

Note that Pmanderson was just blocked for 1 week in an unrelated incident, and therefore won't be able to respond further. —SW—  04:16, 18 March 2011 (UTC)

Stats

I ran an analysis on the first 500 articles from the toolserver list, and found the following statistics:

• 672 distinct references were duplicated at least once.
• A total of 1,820 duplications were detected.
• The maximum number of times a reference was duplicated was 24.
• The average number of times each duplicated reference was duplicated was 2.7.
• 197 of the 500 articles (39%) already had at least one named reference.
  • Out of the 197 articles with named references, the average number of named references per article is 27.1.
  • Out of the 197 articles with named references, the maximum number of named references in an article is 466.
  • 156 articles had 5 or more named references.

—SW—  16:26, 18 March 2011 (UTC)

The usual guideline that we have for these things is that editors (and bots) should not change from one optional style to another (WP:CITEVAR). The use of named references is completely optional, not required; as long as that remains the case, bots should not be changing the citation style. The right place to make changes to our requirements for inline citations is WP:CITE, not in a bot request. — Carl (CBM · talk) 17:52, 18 March 2011 (UTC)

WP:CITE doesn't actually say that it is optional, and Help:Footnotes#Multiple_citations_of_the_same_reference_or_footnote clearly implies that it is not optional. —SW—  17:54, 18 March 2011 (UTC)

• Open Misplaced Pages bot requests for approval