Misplaced Pages

Revision as of 17:41, 29 September 2020

Languages Template‑class

This template is within the scope of WikiProject Languages, a collaborative effort to improve the coverage of languages on Misplaced Pages. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.LanguagesWikipedia:WikiProject LanguagesTemplate:WikiProject Languageslanguage Template This template does not require a rating on Misplaced Pages's content assessment scale.

Writing systems Template‑class

This template falls within the scope of WikiProject Writing systems, a WikiProject interested in improving the encyclopaedic coverage and content of articles relating to writing systems on Misplaced Pages. If you would like to help out, you are welcome to drop by the project page and/or leave a query at the project’s talk page.Writing systemsWikipedia:WikiProject Writing systemsTemplate:WikiProject Writing systemsWriting system Template This template does not require a rating on Misplaced Pages's content assessment scale.

Archives

Archive 1

This page has archives. Sections older than 180 days may be automatically archived by Lowercase sigmabot III when more than 3 sections are present.

This is the talk page for discussing improvements to the ISO 639 name template.

Put new text under old text. Click here to start a new topic. New to Misplaced Pages? Welcome! Learn to edit; get help. Assume good faith Be polite and avoid personal attacks Be welcoming to newcomers Seek dispute resolution if needed

Archives: 1Auto-archiving period: 6 months

Template-protected edit request on 20 September 2018

This edit request has been answered. Set the |answered= or |ans= parameter to no to reactivate your request.

Replace with {{#invoke:ISO 639|name|{{{1}}}}}<noinclude>{{doc|content={{ISO 639 templates|lua=yes}}}}</noinclude> as ISO 639 has been luafied. Using Module:ISO 639 allows for

1) More types of inputs (different types of codes and names)

2) The output of ISO 639-5 names (which should be supported as the name implies)

3) The output of all names officially in ISO 639 (as the names are directly from sil.org)

4) Better organization

– BrandonXLF 03:05, 20 September 2018 (UTC)

Where are the test cases that show that this change won't break anything?

As it stands right now, {{lang}} has a similar database. I had intended to modify this template to use {{lang}}'s data but hadn't yet got round to it. I don't think that en.wiki should have two (or more) databases that hold more-or-less the same thing.

—Trappist the monk (talk) 08:18, 20 September 2018 (UTC)

@Trappist the monk: Module:ISO 639 allows for more inputs then lang. I'll make a test case. I feel as if ISO 639's database and Module:Script databases combined are more complete then lang's. – BrandonXLF 12:28, 20 September 2018 (UTC)

Test-cases at Template:ISO 639 name/testcases – BrandonXLF 12:34, 20 September 2018 (UTC)

{{lang}} has a specific purpose which is to correctly label text using the lang= html attribute. The value assigned to that attribute must be listed in the IANA language-subtag-registry file so that browsers and screen readers understand the meaning of the attribute value. The IANA registry derives from the various ISO 639, 15924, 3166 standards so for the purposes of {{ISO 639 name}}, the {{lang}} data is sufficient. For the enhancements that you propose, {{ISO 639 name}} can get 639-1 names from Module:Language/data/iana languages; can get 639-3 names from Module:Language/data/ISO 639-3. {{lang}} has no use for explicit 639-2 and 639-5 nor for the non-IANA codes from 15924. I would advocate for the creation of separate Module:Language/data/ISO 639-2 and Module:Language/data/ISO 639-5 in the same format as Module:Language/data/ISO 639-3. This lends some amount of efficiency because for the typical case of {{ISO 639 name|<code>}} you search the tables by indexing into them rather than by the cumbersome, time-consuming, brute-force search as is done in Module:ISO_639. For the name-to-code reverse lookup, a separate table could be created by automation that would combine the separate 639-1 data from Module:Language/data/iana languages, and the 639-2, -3, -5 tables to make a table of tables: = {639_1code, 639_2code, 639_3code, 639_5code},.

There is a bug: {{ISO 639 name/sandbox|ang}} → Old English

—Trappist the monk (talk) 13:56, 20 September 2018 (UTC)

@Trappist the monk: Bug fixed, If you want I can move Module:ISO 639/data, Module:ISO 639/data/altnames, Module:ISO_639/data/ISO_639-5 and Module:Script/data to module language replacing the current ones as they have more data, let me know waht you wan, all I really want is a tempalte that does what the name implies (that includes returning ISO 639-5 names. The current sub template is a mess and is very confusing, so it needs to converted to lua one way of another. Also, how is ISO 3166 related to language? – BrandonXLF 19:51, 20 September 2018 (UTC)

 Not done: please make your requested changes to the template's sandbox first; see WP:TESTCASES. BrandonXLF, given that one bug has been uncovered so far the testcases need to show that all the existing uses work, which means all 1678 of the languages used by the existing template in Special:PrefixIndex/Template:ISO 639 name, not the languages in the module's code.

As an aside, (and please, don't take this the wrong way) when you're trying to show that you've been attentive to detail, leaving 3 spelling mistakes & unclosed brackets in your final comment doesn't help your case. Cabayi (talk) 08:57, 21 September 2018 (UTC)

A very large part of the problem with the {{ISO 639 name}} suite of templates is that too many of the 'codes' that they support, aren't ISO 639 codes:

• {{ISO 639 name 1bd}} → {{ISO 639 name|1bd}} : 1bd is a Linguist list identifier so doesn't belong in an ISO template
• {{ISO 639 name Arab}} → Arabic : Arab is four characters so is not an ISO 639 code; is an ISO 15924 script code
• {{ISO 639 name Cherokee}} → Cherokee : Cherokee is a language name; why call a template with a language name to render that same name? (chr → Cherokee)
• {{ISO 639 name Hluw}} → {{ISO 639 name|Hluw}} : Hluw is an ISO 15924 script code
• {{ISO 639 name Ja-Latn}} → Japanese : an ISO 639 code with an ISO 15924 script suffix is no longer an ISO code but is an IETF language tag
• {{ISO 639 name ar-DZ}} → Arabic : an ISO 639 code with an ISO 3166 region suffix is no longer an ISO code but is an IETF language tag
• {{ISO 639 name be-tarask}} → {{ISO 639 name|be-tarask}} : an ISO 639 code with an IANA variant suffix is no longer an ISO code but is an IETF language tag
• {{ISO 639 name be-x-old}} → {{ISO 639 name|be-tarask}} : an ISO 639 code with an IANA private-use suffix is no longer an ISO code but is an IETF language tag
• {{ISO 639 name cbk-zam}} → {{ISO 639 name|cbk-zam}} : presumably cbk = Chavacano and zam = Zamboangueño; ISO 639-3 zam is Miahuatlán Zapotec; zam is not an IANA-registered extlang; cbk-zam is used as a sub-domain name for cbk-zam.wikipedia.org

no doubt there are other non-ISO 639 'codes'. The {{ISO 639 name}} templates that use 15924 script codes should use the {{ISO 15924 name}} template suite; those templates might be upgraded to use Module:Script/data (which should be harmonized with Module:Language/data/iana scripts if possible). For those 'codes' that are in the form of IETF tags, perhaps an {{IETF name}} template that takes its data from the IANA data used by {{lang}}.

—Trappist the monk (talk) 11:25, 21 September 2018 (UTC)

Pretty sure that I did say what it is that I thought should be done.

3166 is the basis for the region part of an IETF language tag: pt-BR → Portuguese as used in Brasil.

—Trappist the monk (talk) 09:03, 21 September 2018 (UTC)

@Trappist the monk: Ok, So how you you want them moved over? I think instead of having ISO 639 name we should just call it language name and have it include IETF/IANA data. So in then end we would have a Script suite and a Language suite with other templates do special formatting to their outputs. Does that sound good? – BrandonXLF 12:24, 21 September 2018 (UTC)

It is not an issue of movement as much as it is an issue of data sourcing and then, once we have decided on sourcing, we must take a decision about how we structure and define that data. Because the data come from multiple sources (infoterm (-1), Library of Congress (-2, -5), sil (-3)), we should attempt to use data directly from the appropriate source. Alas, infoterm apparently doesn't publish the official list of 639-1 codes so we must look elsewhere for them; the likely best place being IANA which prefers -1 over -3 codes when both exist. For -2 and -5, Library of congress publishes lists of codes – the -2 list contains both -2/T and -2/B codes. For -3, sil.org publishes multiple lists that appear to sometimes contain different data for the same code depending on which file you're looking at – for example the current iso-639-3_Name_Index_20180914.tab lists two spellings of aee while iso-639-3_20180914.tab from the same zip file lists only one spelling.

Because {{lang}} already uses 639-1 codes from the IANA language-subtag-registry file and 639-3 codes from sil.org's iso-639-3_Name_Index_YYYYMMDD.tab file, those two are I think, taken care of. For 639-2, it would seem a simple matter to create Module:Language/data/ISO 639-2 from the LOC data in the same format as Module:Language/data/ISO 639-3. For 639-5, Module:ISO_639/data/ISO_639-5 (what is the source for these data?) there are the issues of format conversion and of altnames which do not appear in the LOC listing.

—Trappist the monk (talk) 15:17, 21 September 2018 (UTC)

I've created testcases 1 2 3 & 4 (it's too much for 1 page). There's more that will need work beyond the dialect & script suffixes - bh, eml, & gkm caught my attention. Hope that helps, Cabayi (talk) 15:29, 21 September 2018 (UTC)

That was a bit of work, thanks.

bh is a valid 639-1 code; it is the only two-character collective: {{#invoke:lang|name_from_tag|bh}} → Bihari languages; eml is not a valid ISO 639 code (deprecated 2009); gkm is not currently a valid 639-3 code; see here.

—Trappist the monk (talk) 16:14, 21 September 2018 (UTC)

testcases4 revealed a bug in Module:lang, now fixed. thanks

—Trappist the monk (talk) 18:43, 21 September 2018 (UTC)

The reason that the testcases had to be spread across four pages is revealed here (these parser profiling data captures from testcases2):

CPU time usage	6.248 seconds
Real time usage	6.754 seconds
Preprocessor visited node count	8,038/1,000,000
Preprocessor generated node count	0/1,500,000
Post-expand include size	19,120/2,097,152 bytes
Template argument size	6,948/2,097,152 bytes
Highest expansion depth	5/40
Expensive parser function count	420/500
Unstrip recursion depth	0/20
Unstrip post-expand size	0/5,000,000 bytes
Number of Wikibase entities loaded	0/400
Lua time usage	5.521/10.000 seconds
Lua memory usage	16.14 MB/50 MB

this one from after I added calls into Module:lang (the IETF comments):

CPU time usage	6.048 seconds
Real time usage	6.672 seconds
Preprocessor visited node count	8,430/1,000,000
Preprocessor generated node count	0/1,500,000
Post-expand include size	22,894/2,097,152 bytes
Template argument size	6,948/2,097,152 bytes
Highest expansion depth	5/40
Expensive parser function count	420/500
Unstrip recursion depth	0/20
Unstrip post-expand size	0/5,000,000 bytes
Number of Wikibase entities loaded	0/400
Lua time usage	5.249/10.000 seconds
Lua memory usage	27.76 MB/50 MB

in both cases the expensive parser function count is approaching the limit; once converted to Lua, the count will be zero. The lua time usage metric shows that the brute-force search currently employed by Module:ISO 639 is taking a lot of time; it shouldn't. The lua time measurement is not precise but giving scributo more work to do will not reduce the time it takes to accomplish the original work plus the new work

—Trappist the monk (talk) 18:43, 21 September 2018 (UTC)

@Trappist the monk: I got the ISO 639-5 list form the Misplaced Pages article, I couldn't find any sources. I'll try to find sources for the rest and post them here. – BrandonXLF 19:50, 21 September 2018 (UTC)

Library of Congress is the custodian for -2 and -5. Their lists are:

639-2 codes

639-5 codes

—Trappist the monk (talk) 20:40, 21 September 2018 (UTC)

The more I look at the sub pages the more I'm confused. The naming system is a mess and there's so many sub-pages without documentation it's crazy. I think using Module:ISO 639 for now is a good idea, and then we can merge it with Module:Language. For now we should merge Module:Lang with Module:Language. – BrandonXLF 15:51, 23 September 2018 (UTC)

Yes, the existing {{ISO 639 name}} template suite is confusing; that is why we are having this conversation

I think using Module:ISO 639 for now is a good idea – not in its current implementation for reasons stated above

and then we can merge it with Module:Language – no; the two modules have distinctly separate purposes and functionality

For now we should merge Module:Lang with Module:Language. – no; the two modules have distinctly separate purposes and functionality

—Trappist the monk (talk) 17:25, 23 September 2018 (UTC)

@Trappist the monk: I'm saying Module:Language sub pages are confusing ({{ISO 639 name}} sub pages are confusing as well). I've made changes to Module:ISO 639, how does it run now?– BrandonXLF 20:11, 23 September 2018 (UTC)

BrandonXLF, Have you looked at the testcases 1 2 3 & 4 ?? I think the appearance of Lua error: not enough memory. early in each page answers your question. Cabayi (talk) 20:45, 23 September 2018 (UTC)

... and the testpages have ended up in Category:Pages with script errors. Cabayi (talk) 20:50, 23 September 2018 (UTC)

I have hacked Module:Sandbox/trappist_the_monk/ISO_639_name. Used in Template:ISO 639 name/sandbox is doesn't run out of memory nor time when used in testcases 1 2 3 & 4 and renders error messages for those 'codes' that aren't ISO 639 codes.

I'll add similar code for the {{ISO 639 code-1}}, {{ISO 639 code-2}}, and {{ISO 639 code-3}} templates.

—Trappist the monk (talk) 22:53, 23 September 2018 (UTC)

@Trappist the monk: Your sandbox looks good, but it needs to be able to accept names, it needs to turn AA into aa and it needs to accept ISO 639-5. Also, how's Module:Time/sandbox? – BrandonXLF 04:59, 24 September 2018 (UTC)

The purpose of this template is stated in the first line of its documentation:

The {{ISO 639 name}} template is used to resolve ISO 639-1, ISO 639-2 and ISO 639-3 codes to language names.

A language name is not an ISO 639 code so the template should not accept input that is not a valid 639 code. Alas, because the current template does accept such input, any template replacement must deal with that misuse. I used an error message to show which of the testcases were not valid 639 codes so that something other than the simple error message might be done for them. At the moment I'm thinking that 'codes' that look like ietf tags will be stripped of their subtags and the result used to index into the appropriate 639 table. For non-code input, return the non-code input (perhaps decorated, perhaps not). In both of these cases, the code will add an error category so that these non-codes can be found and fixed.

639-5 will come in its turn; first we must get -1, -2, and -3 to work correctly because those are the codes most in use and supported by the template suite as it exists today.

Can you show a case where my code did not properly handle a valid 639 code regardless of case? The code that you complained about appears to work correctly:

{{ISO 639 name/sandbox|AA}} → Afar

—Trappist the monk (talk) 09:12, 24 September 2018 (UTC)

@Trappist the monk: Are we not trying to make a template that supports {{ISO 639 code-1}}, {{ISO 639 code-2}}, and {{ISO 639 code-3}}? To stay organized it should be the same module. And I don't see why it's and for this to accept names. I can make the lua table for name to code using the find and replace function in Misplaced Pages, should this discussion be moved? – BrandonXLF 22:17, 24 September 2018 (UTC)

Yes, I think that we are trying to make a template that supports {{ISO 639 code-1}}, {{ISO 639 code-2}}, and {{ISO 639 code-3}} as well as the generic {{ISO 639 name}}. And the code that I've written does that:

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_code_1|AA}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_code_1|AA}}

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_code_2|afa}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_code_2|afa}}

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_code_3|chr}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_code_3|chr}}

I agree that all of this should be in the same module (data excepted).

Some words have escaped from your sentence so it no longer makes sense:

And I don't see why it's and for this to accept names.

What did you mean that sentence to say?

Surely there's a better way of constructing a name-to-code table than by search and replace. Why should we consider moving this discussion?

—Trappist the monk (talk) 22:40, 24 September 2018 (UTC)

@Trappist the monk: I meant to say: I don't see why it bad (not and) for this to accept names. And to do the find and replace you find the regex (+"]) = (.+"}) and replace it with the regex $2 = $1,, all you need to do it is AWB. – BrandonXLF 22:44, 24 September 2018 (UTC)

It is pointless for the template to accept a language name that is not an ISO 639 code and return that same language name except as an error condition. Accepting a language name is contrary to the purpose of the template as stated on its documentation page.

—Trappist the monk (talk) 23:05, 24 September 2018 (UTC)

@Trappist the monk: Let's say you're making a template that has a output of a language name at some point. Some people might put the language name as the input for the parameter. Rather the having editors fix these issues, why not let Template:ISO 639 name accept the names of languages? Of course it should be more of a backup. – BrandonXLF 23:11, 24 September 2018 (UTC)

Editors might also give the template the name of the Andalusian ambassador:

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name|Pedro Jiménez de Góngora}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name|Pedro Jiménez de Góngora}}

which is returned with an error message. For this template, language names as input are wrong just as the ambassador's name is wrong.

—Trappist the monk (talk) 14:15, 25 September 2018 (UTC)

Module:Sandbox/trappist_the_monk/ISO_639_name_to_code built by Module:Sandbox/trappist_the_monk/ISO_639_name_to_code/make supports name-to-code:

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Afar|1}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Afar|1}} – 639-1

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Cherokee|2}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Cherokee|2}} – 639-2

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Chazumba Mixtec|3}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Chazumba Mixtec|3}} – 639-3

{{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Chazumba Mixtec}} → {{#invoke:Sandbox/trappist the monk/ISO 639 name|iso_639_name_to_code|Chazumba Mixtec}} – 639 part unspecified

—Trappist the monk (talk) 14:15, 25 September 2018 (UTC)

@Trappist the monk:But it can't change ISO 639-2 to ISO 639-3 for example, But Module:ISO 639/sandbox can.

– BrandonXLF 03:52, 27 September 2018 (UTC)

I don't think that what you are attempting to do is working correctly:

{{#invoke:ISO 639/sandbox|code1|Cherokee}} → {{#invoke:ISO 639/sandbox|code1|Cherokee}}

{{#invoke:ISO 639/sandbox|code2|Cherokee}} → {{#invoke:ISO 639/sandbox|code2|Cherokee}}

{{#invoke:ISO 639/sandbox|code3|Cherokee}} → {{#invoke:ISO 639/sandbox|code3|Cherokee}}

—Trappist the monk (talk) 11:34, 27 September 2018 (UTC)

@Trappist the monk:, my bad. It's fixed now. How does it look? – BrandonXLF 19:44, 27 September 2018 (UTC)

@Trappist the monk: Fixed all other issues too. – BrandonXLF 04:00, 28 September 2018 (UTC)

@Trappist the monk: Can we move the data pages to Module:Language? – BrandonXLF 01:02, 5 October 2018 (UTC)

moving on

... before this discussion becomes moribund and is flushed into an archive

I have usurped Module:ISO 639 name (it was a pseudo redirect to Module:Language/name) and its associated /doc page (a true redirect to Module:Language/name/doc).

This module is currently in use in {{ISO 639 name/sandbox}}. The four testcases pages created by Editor Cabayi are here:

/testcases1

/testcases2

/testcases3

/testcases4

The module has:

Module:Language/data/ISO 639 override to allow us to return an alternate of the standard ISO 639 name when appropriate

support for ISO 639-1, -2, -3, -5; codes and names all taken from custodial or other official sources

support for individual code-part lookup

the ability to link language name to a Misplaced Pages article; to be used by {{ISO 639 name link}}

code-from-name lookup

code to replace expensive {{#ifexists:}} parser function

error messaging (can be hidden by user css) and categorization (Category:ISO 639 name template errors

Given the extensive coverage provided by Editor Cabayi's testcases, I expect little in the way of trouble implementing this module in the wider world. However, now is not the time for feature-creep so no edits should be made to the module except those that repair deficiencies or outright errors.

—Trappist the monk (talk) 14:34, 17 October 2018 (UTC)

I've implemented the module in {{ISO 639 name link}} and in {{User x/doc‎}}.

—Trappist the monk (talk) 16:06, 17 October 2018 (UTC)

Implemented the module in {{Native_name}}.

—Trappist the monk (talk) 13:19, 19 October 2018 (UTC)

Module:Lang is a better fit for {{native name}}.

—Trappist the monk (talk) 10:29, 22 October 2018 (UTC)

Implemented the module in {{Wikisourcelang}}.

—Trappist the monk (talk) 11:00, 23 October 2018 (UTC)

Implemented the module in {{Infobox name module}}.

—Trappist the monk (talk) 13:13, 24 October 2018 (UTC)

Implemented the module in {{Proofreader needed}}, {{User Translator}}

—Trappist the monk (talk) 13:03, 25 October 2018 (UTC)

Template-protected edit request on 18 December 2018

This edit request has been answered. Set the |answered= or |ans= parameter to no to reactivate your request.

We received the folloing rewuest at the help desk:

• please fix {{iso2language|lij}} with link to Ligurian (Romance language). thanks!!! --2001:B07:6442:8903:8D3C:C5DA:57A1:9F2F (talk) 14:53, 18 December 2018 (UTC)

I checked at the ISO639 site: this is the correct mapping. please add:

• lij --> Ligurian (Romance language) Arch dude (talk) 16:08, 18 December 2018 (UTC)

 Already done Already done. Test:

Template:Iso2language

See rest of the discussion at Misplaced Pages:Village_pump_(technical)#template_fix_request. Dreamy Jazz 🎷 22:45, 19 December 2018 (UTC)

• Template-Class language pages
• NA-importance language pages
• WikiProject Languages articles
• Template-Class Writing system pages
• NA-importance Writing system pages