Misplaced Pages

Revision as of 20:13, 27 August 2008

Archive Pre-reflect Reflect

Reflection

Where, exactly, is this reflection taking place? Phil Sandifer (talk) 06:14, 5 July 2008 (UTC)

Fair point, and I need to get a page up explaining what this stage is all about. Essentially, I'm currently compiling a report on all responses to the question phase (circa 200), before submitting it for review for depth of content, balance and neutrality. This report will then form the basis for the recommendation section. It's going to be approximately 2 weeks before the report is up for peer review here, as there's a lot of study to complete. Hope this makes sense. Gazimoff _Read 18:45, 5 July 2008 (UTC)

So this is a solo project, at least in terms of basic report composition? Phil Sandifer (talk) 18:28, 6 July 2008 (UTC)

Ish. Why do you ask? If you could give me some further information or background to your question, I'd probably be able to give you a more complete answer. The structure of the report, however, is based on the Question phase of the review, encompassing all responses provided. Hope this helps. Gazimoff _Read 18:50, 6 July 2008 (UTC)

Because I had been under the mistaken impression that this was a community-driven review, not one person running a survey. Why, on a wiki, are you avoiding a communal writing process for the report? Phil Sandifer (talk) 19:04, 6 July 2008 (UTC)

It's a good question, and a valid one at that. Essentially, I offered to perform this review, as previous wiki-centric processes had failed to reach any meaningful consensus. Because editors hold a wide range of disparate views, it was felt that a survey-based approach would allow editors to contribute their thoughts and feelings without the process becoming bogged down in debate. Besides, the process of compiling the report is much the same as crafting an article - it's quite common for an article to be researched or compiled offline or in a sandbox, before being moved into the mainspace (or projectspace) in this case, once it's in a fit state for wider examination. The process is community driven - the structure of the review, the questions and the responses have all been from the community. All that's happening is that the report is being prepared for upload in one block, so that the community can then analyse it, discuss it and so on in one contiguous piece. The methodology being used is quite similar to that used for corporations and businesses to analyse internal processes. I hope that clarifies things for you. Gazimoff _Read 20:17, 6 July 2008 (UTC)

<- I think, if we had a bigger team and could parcel out the responses 10 or 20 per editor, then a group draft would be workable - and maybe it is. My understanding is that the report from Gazimoff will basically collate the responses and find common threads. For example, on the question "How do you view the role of an administrator?", the report will likely say that, of 200 responses,

1. 77 editors view admins as authority figures
2. 56 editors view admins as janitors
3. 34 editors view admins as enforcers of policy

...and so forth. I would also expect that, once we have this list of responses and how widely held each is, that the community (or Gazimoff, or whomever) would provide some analysis, such as "Examples of admins as policy enforcers included , provided by ]...", where the (admittedly few) diffs would be incorporated. Most of the questions are intended to gauge where RFA and adminship is currently, and I think the specific question I use as an example is intended to give us a target - where is adminship now, and how should that impact how we choose admins? So long as we can edit the report - which we will - I don't have a problem with Gazimoff doing the initiall analysis and tally - though, if he were to provide his reporting format and a block of responses, I'd be happy to pitch in. UltraExactZZ ~ Evidence 15:01, 7 July 2008 (UTC)

That's a goods point, and one that I'm keen on sharing the workload on. My analysis is centred on examining the statements from each editor in order to build up an overall picture, and is time consuming. If you can perform a statistical analysis on the feedback, the report can contain both a mathematical and logical analysis of the survey results. It's a great idea, and I'm all for it. Many thanks! Gazimoff _Read 10:22, 8 July 2008 (UTC)

OK, let me work up a method, then. Anyone else want to pitch in? UltraExactZZ ~ Evidence 12:40, 8 July 2008 (UTC)

I'm game, assuming you guys have any use for relatively unskilled !labor. I've also got access to a dumb station to run things on I won't need for the rest of this week. Cheers. --lifebaka (Talk - Contribs) 20:28, 8 July 2008 (UTC)

I've created a category for the unsubmitted responses, Category:Unsubmitted Wikipedian responses to RfA Review, and my first task will be to move the unsubmitted items from the main Category:Wikipedian responses to RfA Review to the unsubmitted category. Then, the responses category will match the list at Misplaced Pages:RfA Review/Question/Responses, except that it will be in alphabetical order. Then I'll break the list into 20-25 names apiece, and each of us can read through the responses and tally the comments. I don't think we need to go too in depth, as that's what Gazimoff is doing with his logical analysis. This read will be to determine, for example, how many people oppose Self-Noms? I'll plug the tallys into a spreadsheet and graph the results. Maybe I'll do a trial run tonight, just to see if it's feasible, but I don't anticipate many problems. UltraExactZZ ~ Evidence 20:51, 8 July 2008 (UTC)

That's great. I'd like to make a judgement call on completed response sheets that were not added to the responses list but are still in the cat, as there is a lot of feedback in some of them. Personally, I'd like to include all apart from the blanks and non-response-related submissions. Also, if possible I'd like some stats on the responders, such as admin/non-admin, time since registered etc as a form of demographic analysis, although I suspect this would be best suited to bot-work. I also have a cat, Category:Processed responses to RfA Review, which I'm using to tag responses once I've processed them. You might want to look at a similar cat to track tally processing. You might also want to define your tally chart before counting, so that we can get an idea of the granularity. Hope this helps, bit of a rush of blood to the head :) Gazimoff _Read 21:35, 8 July 2008 (UTC)

I agree, except that I would like to leave the chart open until we're done. If I have three different responses for Question 1, and the next survey has a response that fits none of those, then I can just add it to the list; subsequent items that match it can be added to that tally. I think I moved two unsubmitted responses, mainly to populate the category - but, I agree, only blanks should go there. UltraExactZZ ~ Evidence 21:49, 8 July 2008 (UTC)

Fair enough, works for me. If I come across any blanks myself, I'll shift them into your cat as well. Many thanks, Gazimoff _Read 22:06, 8 July 2008 (UTC)

Opening this up

After some discussion, I'm opening this work up. i was wrong to think I could complete it by myself, and I need help piecing it all together. currently I have handwritten notes on some 60 submissions, which I'm using to create the article structure before filling them out with information from the remainder. Problem is, at this rate it's going to be about 3 months before I manage to process the remaining three quarters of the submissions. That's why I need help in finishing this article.

All submitted responses should be in Category:Wikipedian responses to RfA Review. Processed responses should also be in Category:Processed responses to RfA Review. Blank or meaningless forms should be in Category:Unsubmitted Wikipedian responses to RfA Review. What needs to happen is for each response that has not been processed, responses to each section need to be analysed to see if they add anything to the report at WP:RREV/R, or help to clarify meanings. If they include refs to examples, add those refs to the reflect article. Once that's done, tag it as processed and move on to the next one. Once all are processed, we can develop conclusions and summary bullet points. One of the things that UltraExactZZ was looking at was adding tally information to give an idea of the level of responses for each section of the report. If you'd like to help out in this area as well, that would be great.

Many thanks for any help and support you can provide in completing this report. Gazimoff _Read 00:46, 18 July 2008 (UTC)

I'll be throwing some time at this, to see what we can extract from it and some pointers as to where we are headed, and we will be pleased to assist in whatever way I can. Pedro :  Chat  07:42, 18 July 2008 (UTC)

If you'd be interested in helping me process responses and build the report, that would be a huge hlp. It's time consuming work, but hopefully worthwhile. Many thanks! Gazimoff _Read 08:32, 18 July 2008 (UTC)

I'm working my way through a tally of how many times common responses were given. Once I have fifty or so, I'll post something and group the remaining responses, so others can tally as well. I'm basically adding unique responses to the list as I go, but those are getting fewer and farther between as I proceed. One thing I can say with certainty, though - almost everyone agrees that candidates should be able to withdraw at will. UltraExactZZ ~ Evidence 15:19, 18 July 2008 (UTC)

Breaking for lunch, I'm up to 36; I expect I'll have 50 by the end of the day. UltraExactZZ ~ Evidence 15:28, 18 July 2008 (UTC)

Tally Assistance

OK, there are two phases of the Reflect process. One is the Qualitative analysis, which Gazimoff and others are working on - what was said in the responses? The other is the Quantitative analysis, which I am working on - how many people said X, Y, or Z? I stress that this doesn't dimish the responses into sheer numbers, or a popular vote - rather, it's intended to highlight numerical trends in the responses to help find common threads, and to determine how common those threads were.

Here's where I am. I've taken the first 55 or so responses (alphabetically) and tallyed the comments. The different comments I found are broken out by question at User:Ultraexactzz/RREV Tallys. What I'd like to do, if this format works, is to set up a page for each set of remaining responses, 5 at a time, and get others to assist in tallying the responses. If there's a comment that doesn't fit a current field, no problem - a new row can easily be added. Once we have them all, I'll combine them and make up nice pretty graphs of the results.

Is this a process that can work? Or is there a better way I'm missing? Thanks in advance for the assistance, UltraExactZZ ~ Evidence 18:14, 24 July 2008 (UTC)

It looks reasonable enough to me. No problems here. Gazimoff_{(mentor/review)} 18:24, 24 July 2008 (UTC)

 Done I have posted the first 100 responses in ranges of 5 each at Misplaced Pages:RfA Review/Reflect/Statistical Analysis, and will complete the list later today. It should be pretty straightforward - and thanks to everyone who helps out! UltraExactZZ ~ Evidence 14:54, 25 July 2008 (UTC)

I'll take a stab at some of the "groups of 5". Can statistics really be fun? I'll report back.  :) Livitup (talk) 15:52, 25 July 2008 (UTC)

Dunno' what's goin' on with this right now (nothing since the 25th, looks like), but I'll be doin' at least one a day 'til they're all done. Cheers. lifebaka (talk - contribs) 04:07, 31 July 2008 (UTC)

Life intervened, I'm afraid - but I've now posted the remaining 15 lists, so all responses are accounted for. I'll work a list later today; with enough help, we should bang this out in no time. UltraExactZZ ~ Evidence 12:43, 31 July 2008 (UTC)

I did one of the ranges. Wow, was it tedious. I mean fun. It was fun. Join in, everyone! Useight (talk) 18:46, 31 July 2008 (UTC)

m While the plural of Majorly might be Majorlys, I'm pretty sure that the plural of tally is tallies. SHEFFIELDSTEEL 21:23, 31 July 2008 (UTC)

Thanks again for the assistance, we're making good progress. One note, though - if you find a questionnaire that's blank, just take the "1" out of the "Total Responses" line at the top of the page, and leave a note at the bottom or in your edit summary; I'll remove it from the total, and from the category for responses. That'll help when we do totals, such as finding out how many responses favored self-noms vs the total, or what have you. Thanks again for the assistance! UltraExactZZ ~ Evidence 14:59, 4 August 2008 (UTC)

Livitup's doing the last one, then we are  Done with analysis. UltraExactZZ still has three to record, though. Cheers, everyone, and great work. lifebaka++ 13:11, 13 August 2008 (UTC)

Nice job, guys. That was some tedious and time consuming work. Well done. Useight (talk) 14:46, 13 August 2008 (UTC)

We're finished!

And... I'm spent. Turns out statistics can, indeed, be fun. :) Livitup (talk) 13:33, 13 August 2008 (UTC)

Thanks to everyone who helped out - we now have statistics on all 209 responses. I'm going to see if there's a way to get this spreadsheet (which includes over 440 distinct responses and statements) into some sort of format to distribute, if only to ensure that this review is open and public. Maybe I can post the CSV as code, perhaps? We'll figure it out, but for now I'll be posting some of the highlights. Again, thanks to everyone for helping! UltraExactZZ ~ Evidence 15:33, 13 August 2008 (UTC)

Range 30

On range 30, Tony1 chose to make comments outside of the questions. I'm not quite sure how to handle this, though my first blush idea would be to look at what he said and just put it in the question-blocks where it fits. Just trying to get a little advice first, though. Cheers, everyone, and thanks. lifebaka++ 14:42, 12 August 2008 (UTC)

Range 28 had someone do the same, and I just plugged in the answers where they fit. I've also been doing that for the last question, if they offer a suggestion that I know I've seen in a previous question - statements about NOTNOW falling under voting, for example.UltraExactZZ ~ Evidence 14:55, 12 August 2008 (UTC)

Gotcha'. I'll go do it, then. Cheers. lifebaka++ 14:57, 12 August 2008 (UTC)

Tinkering with Graphs

OK, so now that we have our numbers, I'm starting to see what graphs would be worth including. Are there any comparisons you see that might be interesting? I have a few in mind already. I'd also like to know if it's worthwhile to put graphs in the Reflect page itself, or if an appendix (or Graph gallery) would be better. Thoughts? UltraExactZZ ~ Evidence 16:48, 13 August 2008 (UTC)

I think the 3D on some of the graphs isn't necessary, but it's not a very big issue. So far I like what you've been doing. A pie graph for C4 between "limited canvassing OK"/"link from userpage OK", "unlimited canvassing OK", "no canvassing OK", and "bot generated list OK" might be useful. On the bar graphs, noting total responses to each of the questions would be useful for interpreting the data; this could be included in the footnote, though. Just some thoughts. Cheers. lifebaka++ 18:25, 13 August 2008 (UTC)

Side note, I can't tell the color scheme for 1% and 30% they are too close to each other. Also, how do we end up with 126%?---Balloonman 21:32, 13 August 2008 (UTC)

Ehm, I believe it's number of users, not percent of users. So, 126 is a reasonable number out of 209. A bit low, if anything. Cheers. lifebaka++ 21:37, 13 August 2008 (UTC)

Yeah, that's total responses, which I should indicate somewhere. I'm mainly trying to show which statements popped up more than others, but there are a few snags. For example, we had a few people who said that "RfAs don't get enough attention" AND "Limited Canvassing is OK, if neutral", so when we graph canvassing responses, that person is counted twice - they held both sentiments. So percentages are dicey, and will be for most questions. The exceptions will be the next to last two, "Ever vote in an RfA?" and "Ever a candidate?", where we have a sharp divide between yes and no. For those, I'll add the responses where the editor didn't offer an answer as a third choice, "No response", and give a percentage of the total. I think I already did one already for "Ever voted?". UltraExactZZ ~ Evidence 01:52, 14 August 2008 (UTC)

For my own preference, I'd avoid 3D graphs as they can introduce a perspective distortion in the representation. Other than that, it's some fantastic work and given me a bit of a kick up the ass to get my side finished :) Gazimoff 10:19, 14 August 2008 (UTC)

OK, we have stats for every section, as well as some prose to explain what the numbers are. I'd like to get some input on what else to add to the report. Are there RfA statistics that would be worthwhile? I think we're closing in on the next phase, which should be exciting. UltraExactZZ ~ Evidence 15:03, 20 August 2008 (UTC)

I was thinking that there's a bot that parses a list of active admins (currently around the 1000 mark). I'd like if possible to get some recent trend graphs to show how the number of total admins and number of active admins vs total registered user count. What do you think? Gazimoff 15:07, 20 August 2008 (UTC)

Total admin counts are listed at User:NoSeptember/admincount, which would be fun to graph. Still looking for the active list, though - but I agree, we should include it as background. UltraExactZZ ~ Evidence 15:20, 20 August 2008 (UTC)

Admin statistics, Inactive and Active

OK, so I've got the numbers for active admins, based on trolling the update history of the master User:Rick Bot updated list at WP:LA. Near as I can tell, automatic updates started on 20 August (!), so we have a year of data. I'll run the inactive numbers and compare the trend for the past year, unless someone has stats for more than a year ago... maybe? UltraExactZZ ~ Evidence 12:21, 21 August 2008 (UTC)

I dont have one myself. But I think it'd be great to see that graph. You might also want to look at User:E/AdminStats for further background information, particularly the part on total admin actions. Hope it's of interest to you. Gazimoff 12:29, 21 August 2008 (UTC)

I'll have a look. I've got the graph for the past year, though - I'll upload it momentarily. It looks like adminship has trended up (from 1307 last august to 1584 now), while active admins have stayed relatively flat. Let me see what I can come up with. UltraExactZZ ~ Evidence 13:02, 21 August 2008 (UTC)

Here's what I have for the past year:

• Total Admins
• Proportion of Active to Inactive
• Active vs Total, Trendlines

There are also some charts at User:NoSeptember/admin graphs, including Image:En-admin-growth.png, which is exactly what I need - but it doesn't have April - July 2007, which I'm missing. If I can track those numbers down, or extrapolate them, we'll have a 6 year trend to review. UltraExactZZ ~ Evidence 14:04, 21 August 2008 (UTC)

That's some fantastic work. So essentially, for the previous twelve months, although the total admin count has increased the number of active admins has flattened? I think in order to round this work off, I need to take a copy of the data table that lists the admin actions undertaken by each admin and do some analysis there for another graph in order to examine the admin workload breakdown. I'll let you know once I have more on that. Great work so far though! Gazimoff 14:46, 21 August 2008 (UTC)

It is good work, but if I can impose here just for a small, yet important point (given that I haven't contributed before here), the labelling as given in the description for the thord graph does not correspond to what is shown. The graph shows numbers of people, and the description states "This chart uses data from the List of Administrators and other sources to show the proportion of active administrators to total administrators." which uses "proportions". It may seem a small point, yet it is highly important in the field of interpretation of data, of which others have stated elswhere I have some expert knowledge.

The two (the graph and its description) need to be brought into agreement: Either (a) change the description to use "numbers" instead of "proportions", or (b) recast the graph so that it plots the proportion of active admins against time, there being no need to have the proportion of admins in this case, as it would always be at 100% or 1.0 (depending on whether one uses percentages or fractional depictions of proportions), or (c) change the wording in this graph to use "numbers" and plot a small additional graph which directly plots the proportion.

It depends on whether one wants to interpret the figures as indicating that "the total number of admins has risen, whilst the number of active admins has flattened", or whether "the proportion of active admins has decreased"

If one were really intent on exploring this, and it may be too much to realistically do, one might consider doing some log-linear modelling to assess any statistical significance to such data, though the information would have to be re-jigged to do that (issues to do with satisfying various crietria to make the applied log-linear models valid, etc)  DDStretch  (talk) 15:01, 21 August 2008 (UTC)

Yep, you're completely right about the third chart and I'd add to it that not starting the axis at zero makes it even harder to judge. Mind you, the second one covers the proportion of active to inactive easily enough, while the first one shows total trends easily enough. I'm not sure if any log-lin analysis would help here - the dataset doesn't feel as if it would show anything from it and I'm not sure if it would highlight anything that would be of use. I'm happy to listen to suggestions though. Gazimoff 16:05, 21 August 2008 (UTC)

This was the chart I was thinking of. The data behind it (you can read it off the chart) shows that roughly 80 admins are responsible for 50% of admin actions (deletes, blocks, protects, granting rollback/acc, etc). Slicing it a different way, roughly two thirds of the admins have performed 10% of the total admin actions, while one third are responsible for 90% of the total number of actions. I think this chart shows that not only do we need more admins, but we need more active admins willing to perform janitorial duties. Gazimoff 15:50, 21 August 2008 (UTC)

I've updated the third graph, above, with a clearer description that substitutes numbers for proportions - and remind me never to upload images when I'm in a hurry, as that's what I get for copy-pasting. My intent was to illustrate the trends in both total and active admins, and didn't think that starting at 600 would cause a problem (as neither figure dipped below 800 during the sample period). I can correct that bit, as well, but I concur with Gazimoff that the first two graphs accomplish our purposes well enough. UltraExactZZ ~ Evidence 16:43, 21 August 2008 (UTC)

Recommend

I know we want to add a section offering background on adminship - sort of a where we are, where we've been section - but what other prose items do we need to add? I'm reading through, and wondering if we can kill two birds with one survey by drawing up conclusions and using those conclusions as the basis for the Recommend phase survey. Thoughts? UltraExactZZ ~ Evidence 16:19, 25 August 2008 (UTC)

That's exactly the plan. We should be able to detail conclusions from the current process, and use those as the basis for constructing recommendations. On that note though, what do you think the conclusions are? Also, I'd like to do some images for the perceptions/attributes element, in order to illustrate these topics. Hope this helps, Gazimoff 12:35, 26 August 2008 (UTC)

We've got a graph for each, listing top responses. Are their any contradictory pairs you'd like to compare, or do you have something else in mind for those images? UltraExactZZ ~ Evidence 13:02, 27 August 2008 (UTC)

<- I think we will want to branch out in two directions. The Reflect Report should end with statements such as this:

• "There is evidence that editors see RfA as a negative process, with issues such as badgering of opposes, personal attacks, and a general lack of civility being prominent concerns."

For the recommendation questionnaire, We'd frame the question this way:

• "There is evidence that editors see RfA as a negative process, with issues such as badgering of opposes, personal attacks, and a general lack of civility being prominent concerns. How would you recommend changing the RfA process to address these concerns?"

We'll either get concrete so-fix-it style responses, with policy proposals that could be turned into policy, or we'll get "Once the culture of RfA changes, these problems will go away", or we'll get "I don't see those concerns as a problem." UltraExactZZ ~ Evidence 12:59, 27 August 2008 (UTC)

I agree with this. I also think we should specifically look at areas that contributors have stated doesn't work well for them, and try to address them by building up a set of requirements to initiate the recommend phase with. Areas that people feel work well can have less of a focus. Hope this makes sense Gazimoff 18:51, 27 August 2008 (UTC)

Misplaced Pages:Perennial proposals

I'm seeing several recommendations from the RfA Review that are listed as Perennial Proposals. For example, 16 editors suggested that admins should stand for reconfirmation periodically - and this is listed as a perennial proposal, which fails largely due to the scale of the task (over 25 reconfirmations per week for the 1,500+ admins currently on the list). Do we still forward this as a proposal? Or do we cite WP:PEREN, and go in a different direction? De-bundling the tools is on there as well, with its failure cited as a technical issue. Is that worth revisiting? UltraExactZZ ~ Evidence 14:24, 27 August 2008 (UTC)

I say it is worth revisting. Perhaps if a number of editors still think it's a good idea, such as the 16 mentioned above, maybe some changes could be used to the alter the idea to make it more feasible, such as making the reconfirmation RFA only every two years or only applicable to "active" admins, in order to cut down on the load. Since "information is king" I think as many proposals as possible would be the best. Useight (talk) 16:31, 27 August 2008 (UTC)

I would agree, though I think we should give as few details as possible, so that the meat of the proposal comes from the editors. Maybe something like:

• "Some editors favor requiring admins to have their administrator status confirmed periodically, in order to ensure that they have the trust of the community. What do you think? How often should such reconfirmations take place? How complex - or simple - should the process for reconfirmation be?"

I think that would hit the highlights. It also leaves as much as possible to the editor. Thoughts? UltraExactZZ ~ Evidence 18:59, 27 August 2008 (UTC)

Discussion of negative consequences of failure?

On my RfA Review page, I said:

...I believe there are many people who hang out at WP:RfA would can tell with 80% or 90% accuracy whether a candidate is likely to pass or not. It would save the time of people at WP:RfA, improve the quality of their advice (since they wouldn't have as much work to do), and save the time and feelings of the candidates, if people would hold their fire, and the process would start with someone knowledgeable saying something like "Please look at the following 3 previous candidates whose contributions remind me of yours...notice they didn't succeed, and consider the advice they got. Are you sure you want to do this now, or would you like to prepare a bit first?" The question should only come from someone who has a track record of making the right call and of having the best interests of the candidate at heart; asking a candidate to withdraw for any other reason is like to either be or be perceived as bullying.

I haven't seen discussion on other pages about the negative consequences of failure. If someone thinks they'll do great at RfA if they just do X, Y and Z, and they spend 3 months jumping through what they think are the right hoops, and fail, it's likely to sour them on Misplaced Pages. We don't need to say "You can't run for RfA unless you've been active for 6 months", but having a few people act in the role of "trusted advocates", with the job of giving sound advice to any candidate about their chances based on the track record of similar candidates at RfA, would probably streamline the process, save people's time, and help morale all around. - Dan Dank55 (send/receive) 16:23, 27 August 2008 (UTC)

But who will be on this "filtering" team and how do we decide who goes on the team? Does this extra layer actually provide enough benefit to justify its creation? Useight (talk) 16:41, 27 August 2008 (UTC)

Some of that occurs at Admin Coaching, with coaches themselves evaluating candidates. This is good and bad; it creates that mentor relationship, but it also focuses the coaching on RfA, which is problematic. That's one of the criticisms of self-noms, that no one has vetted a candidate before they go to RfA. In theory, a nominator has reviewed the candidate's work and thinks that they can pass an RfA. UltraExactZZ ~ Evidence 16:54, 27 August 2008 (UTC)

First, this is what the nom shold be doing. The nom should be asking 1) Will the candidate make a good admin? 2) In the current environment, will the candidate pass an RfA? But, let's ignore the nom. Your proposal is, in part, what admin coaching is intended to do. If a coachee fails an RfA, then I put the blame for that failure on the Coach not the candidate. (Conversely, if the coachee passes an RfA, I put the credit for passing on the candidate.) Unfortunately, we've had a large number of coachees that failed in May/June that gave coaching a black eye. This is a work in progress but I was taking a look at what caused coaching to fail a few months ago. It failed because suddenly people started treating coaching as a "get by RfA" pass and rushed candidates through the process and not truly vetting them... or the candidate pushed the envelope and decided to run without their coaches support. I think some people do a better job at vetting candidates than others. Some people can predict the fate of an RfA better than another, but you can never know. I like to think I vet candidates pretty well and have a sense of the way the wind is blowing. Many of my candidates/coachees get 100+ supports because I don't nom people I don't think will be good admins and with one notable exception I don't nom them if I don't think they will pass. That being said, I recently had a candidate get SNOWED on me... I THOUGHT he would pass, but his RfA didn't even last a day! You never really know what is going to happen.---Balloonman 17:07, 27 August 2008 (UTC)

Agreed - we had several editors criticize coaching as "Teaching for the Test". You know, it might not be a bad idea to do a mini-RfA Review with the 5 or 10 best coaches and see what they're doing right, then use that as fodder to expand/formalize/rennovate the current coaching process. Best Practices are usually called that for a reason, after all. UltraExactZZ ~ Evidence 17:24, 27 August 2008 (UTC)

Actually, that is what I am kind of working on... reviewing what is being done... and who is doing it. I know that there are some people whose coaching I DO NOT respect. There are others who do a good job. I'm happy when the big criticism for my coachees is that they "might be over prepared." It means that I did a good job.---Balloonman 17:30, 27 August 2008 (UTC)

My fault, I'm not being clear, I'm talking about self-noms. We currently have 3 open self-nommed RfAs, all of which are on the minus side of 70%. All of these guys chose, for whatever reason, to go it alone. I don't want to say there's only one system that would work, because I'd be wrong, but just for instance, suppose that everyone has the "right" (if they're moving in the right direction) to get assigned to an advisor, who will give them a quick assessment every couple of months until they're ready...nothing too exhausting or too fancy, just "tell me what you've done good and bad...nope, look at these 3 other noms, their contributions remind me of yours, and they didn't make it...look for yourself and see if you agree; you may want to study them and see what went wrong, but if you think you can make it anyway, go ahead"...that kind of thing. Suppose we have a sign-up sheet for this role of advisor, and take only the first 10 volunteers (because if it's a long list, no one advisor will be able to establish the kind of track record of impartiality that would lead skittish candidates to trust them, although the list of 10 might change over time). At the top of WP:RfA, we have an infobox that says, if you want to self-nom, here's a list of people who have a good track record in this kind of thing. (My guess is it would work better to assign people to advisors, otherwise potential candidates will think...wrongly..."Oh, Balloonman is famous, I'll do a lot better if I pick him".) Granted, if the candidate doesn't give honest information to the guy they're relying on, the results will be bad; you can lead a horse to water, but you can't make them drink. This is another reason for a more lightweight advisor–advisee relationship than what's available now; it doesn't suck up too much of the advisor's time with a candidate who might be less than honest. - Dan Dank55 (send/receive) 18:02, 27 August 2008 (UTC)

Some would say I'm infamous... not famous...---Balloonman 18:11, 27 August 2008 (UTC)

I'm not sure that the advisor role couldn't be filled by a coach, if only as a form of pre-coaching. The caveat there is that we're already short on coaches, so adding more burden to the process as-is would be problematic. An alternative might be to show the last 20 RfAs with self-noms, and the last 20 RfAs with regular nominations. These lists would show the editor's editcounts, length of experience, and support/oppose %. This might be a good way to illustrate minimums without actually debating and setting them; the average of the last 20 RfAs would be given in terms of successful and unsuccessful, so editors considering it could see the trends. if they're below the average for successful RfAs, they might consider waiting. Oh hell, I'm going to have to do stats on this, aren't I? UltraExactZZ ~ Evidence 18:31, 27 August 2008 (UTC)

←Absolutely! I complained about style guidelines, and all it got me was the "honor" of doing the monthly WT:UPDATES :). That's actually not a bad idea; people would trust statistics even more than they'd trust an advisor. Excellent approach. - Dan Dank55 (send/receive) 18:46, 27 August 2008 (UTC)

I also strongly encourage editors to try out Editor Review before going for RfA. It might identify problem areas or pitfalls beforehand. It might also gain you interest from an experienced editor or administrator willing to nominate you for adminship at some stage in the future, or who would be prepared to coach you. And Ultra, while I'd love to help you out on the coaching side of things, I'm still way to inexperienced as an admin to be useful there. I'm happy to perform editor reviews and assesments though inorder to provide people with useful feedback. Hope this helps, Gazimoff 18:57, 27 August 2008 (UTC)

And maybe Editor Review is something that can provide insight into coaching and pre-adminship mentoring... Maybe there are best practices there that would work well elsewhere. Too... many.... ideas.... UltraExactZZ ~ Evidence 19:35, 27 August 2008 (UTC)

As a point of information, based on the last 25 successful Candidates for Adminship (from Lifebaka to Ice Cold Beer), the average support ratio is 94.08% (Support/Total, discounting Neutrals). The average editcount at the beginning of the RfA was 10963. If we take J.delanoy and Good Ol'factory out, both of whom had well over 45000 edits at the start of their RfAs, the average is still 8003. The average age of account for each candidate, as measured from first edit to the RfA being transcluded, is 22 months. Haven't looked at Unsuccessfuls yet, but I suddenly found this fascinating, so I thought I'd post it. UltraExactZZ ~ Evidence 19:35, 27 August 2008 (UTC)

More details - The range of editcounts was 45000 on the high end (noting that the actual counts are much higher) to 2974 on the low end. Account age ranged from 45 months to 2 editors with only 5 months of experience. The averages for unsuccessful candidates are actually not that different from the successful candidates; The average edit count for the last 23 unsuccessful RfAs (not counting NOTNOW closures, but counting SNOW) was 7495 edits. Average account age was 19 months. UltraExactZZ ~ Evidence 20:13, 27 August 2008 (UTC)

• RfA Review