Wikimedia projects by dump size, and average compressed article size
I was looking at the list of Wikipedias, and I thought that after all, the number of articles is a very bad measure of the size of a Wikipedia, because articles can have various sizes. More specifically, some Wikipedias have bots which create stubs about all sorts of villages and other geographic locations (to take the random example of the Volapük Wikipedia), and this makes them seem bigger than they really are.
It seems fairer to compare the dump size of the wiki as a measure of the total quantity of text, or rather the compressed dump size to eliminate redundancy. Surprisingly, I didn't manage to find this information in a usable chart, so here we go... Note that I'm speaking about the dumps from the Wikimedia dump service, and more specifically the pages-articles.xml.bz2 dump. You might want to look precisely what this measures (it does not include images or page history, for instance).
curl -s 'http://dumps.wikimedia.org/backup-index.html' |
grep '<li>' |
grep '<a href' |
cut -d '"' -f 2 |
cut -d '/' -f 1 |
sort |
uniq |
while read p; do
echo -n "$p "
curl -s "http://dumps.wikimedia.org/$p/latest/" |
grep 'pages-articles.xml.bz2"' |
head -1 |
cut -d '>' -f 9 |
cut -d '<' -f 1
sleep 1
done |
awk '
/K$/ {printf "%s %d\n", $1, $2*1024}
/M$/ {printf "%s %d\n", $1, $2*1024*1024}
/G$/ {printf "%s %d\n", $1, $2*1024*1024*1024}' |
sort -k2,2nr
Yes, I know, this is really ugly and will break the day Wikimedia changes the layout of the dump service pages in the slightest way, but it works for now. It no longer does. Here is the current result:
enwiki 7301444403 dewiki 2040109465 frwiki 1503238553 commonswiki 1288490188 jawiki 1288490188 itwiki 1181116006 eswiki 1073741824 enwikisource 1041026252 ruwiki 966367641 plwiki 823866163 ptwiki 697093324 nlwiki 647495680 frwikisource 604294348 zhwiki 538548633 svwiki 371195904 cawiki 361863577 huwiki 357878988 zhwikisource 335439462 fiwiki 321493401 ukwiki 311531929 cswiki 297166438 dewikisource 295908147 nowiki 295698432 hewiki 250295091 trwiki 209924915 kowiki 208456908 enwiktionary 203318886 ruwikisource 201431449 viwiki 181403648 arwiki 178782208 rowiki 167981875 dawiki 155923251 idwiki 153826099 srwiki 149631795 bgwiki 147115212 eswikisource 126877696 fawiki 124256256 hrwiki 124256256 arwikisource 122788249 eowiki 118174515 frwiktionary 116706508 skwiki 114294784 thwiki 111463628 ltwiki 103704166 elwiki 102655590 slwiki 101711872 enwikibooks 88814387 itwikisource 88394956 mswiki 82522931 glwiki 82313216 hewikisource 80530636 etwiki 76860620 euwiki 75916902 simplewiki 69310873 shwiki 67423436 hiwiki 66794291 mkwiki 64067993 nnwiki 61236838 plwikisource 60293120 en_labswikimedia 58195968 metawiki 55469670 enwikiquote 54525952 kawiki 50226790 zhwiktionary 50226790 lvwiki 49073356 ptwikisource 47500492 tawiki 47290777 tewiki 45193625 slwikisource 44459622 ruwiktionary 43096473 azwiki 41418752 lawiki 40684748 dewiktionary 40160460 flaggedrevs_labswikimedia 39950745 readerfeedback_labswikimedia 39950745 specieswiki 38692454 bswiki 37224448 knwiki 37224448 dewikibooks 36595302 be_x_oldwiki 34603008 mlwiki 32820428 sqwiki 32505856 brwiki 31771852 plwiktionary 31142707 enwikinews 30513561 bnwiki 30303846 enwikiversity 30094131 iswiki 29150412 kowikisource 28206694 sourceswiki 28206694 cswikisource 27787264 svwikisource 27472691 bewiki 26738688 bgwiktionary 26528972 afwiki 26109542 cywiki 26004684 lbwiki 25794969 huwikisource 25165824 tlwiki 24117248 anwiki 23488102 nowikisource 23278387 ptwiktionary 22858956 lawikisource 22334668 ocwiki 22229811 mrwiki 22124953 srwikinews 21390950 mgwiktionary 20866662 viwiktionary 20552089 fiwiktionary 18874368 elwikisource 18664652 itwikiquote 18454937 vowiki 18350080 rowikisource 18245222 fywiki 18035507 huwiktionary 18035507 thwikisource 17930649 alswiki 17511219 ltwiktionary 17511219 swwiki 17196646 hrwikisource 17091788 jvwiki 17091788 zh_yuewiki 16882073 ndswiki 16672358 siwikibooks 16252928 frwikibooks 16148070 guwiki 16148070 incubatorwiki 16148070 eswiktionary 16043212 urwiki 15938355 nostalgiawiki 15728640 srwikisource 15623782 hywiki 15414067 siwiki 15309209 nlwikisource 14994636 itwikibooks 14889779 trwiktionary 14365491 fawikisource 13946060 astwiki 13841203 elwiktionary 13736345 kowiktionary 13736345 warwiki 13421772 nlwiktionary 13316915 fiwikisource 13212057 gawiki 13107200 mediawikiwiki 13002342 kkwiki 12687769 iowiktionary 12582912 jawiktionary 12268339 eswikibooks 12058624 plwikiquote 12058624 trwikisource 11953766 itwiktionary 11744051 dewikinews 11639193 lmowiki 11324620 tawiktionary 10800332 huwikibooks 10695475 frwikinews 10590617 scnwiki 10590617 jawikisource 10380902 barwiki 10276044 svwiktionary 10276044 iowiki 10171187 mywiki 10066329 iawiktionary 9961472 ukwikisource 9961472 suwiki 9856614 mnwiki 9751756 viwikisource 9542041 cvwiki 9437184 dewikiversity 9437184 quwiki 9437184 ptwikibooks 9332326 newwiki 9017753 yowiki 9017753 arzwiki 8912896 kuwiki 8912896 pmswiki 8912896 plwikinews 8808038 nlwikibooks 8598323 ttwiki 8493465 yiwiki 8388608 tawikisource 8283750 vecwiki 8178892 liwiki 8074035 newiki 7969177 cebwiki 7759462 itwikinews 7759462 plwikibooks 7654604 frwikiversity 7549747 kuwiktionary 7549747 bnwikisource 7444889 dawikisource 7444889 etwiktionary 7235174 mywiktionary 7235174 pswiki 7235174 tewikisource 7235174 eswikiquote 7130316 pamwiki 7025459 scowiki 6920601 uzwiki 6920601 eswikinews 6815744 ruwikiquote 6815744 de_labswikimedia 6710886 zh_classicalwiki 6710886 wawiki 6606028 ruwikibooks 6501171 idwikisource 6396313 jawikibooks 6186598 oswiki 6081740 dewikiquote 5976883 napwiki 5872025 ptwikinews 5872025 zh_min_nanwiki 5872025 hywikisource 5767168 sahwiki 5767168 htwiki 5662310 mtwiki 5662310 sqwikibooks 5662310 cawikisource 5557452 iawiki 5557452 nowiktionary 5557452 bpywiki 5452595 gvwiki 5452595 mwlwiki 5452595 strategywiki 5452595 hsbwiki 5347737 nds_nlwiki 5347737 rowiktionary 5347737 hewikibooks 5138022 ptwikiquote 5138022 vlswiki 5138022 gdwiki 5033164 tgwiki 4928307 foundationwiki 4823449 knwiktionary 4823449 pnbwiki 4823449 kmwiki 4718592 mlwiktionary 4613734 frwikiquote 4508876 jawikinews 4508876 tewiktionary 4404019 betawikiversity 4299161 cswiktionary 4299161 roa_tarawiki 4299161 ukwiktionary 4299161 bat_smgwiki 4194304 fowiki 4194304 itwikiversity 4089446 mlwikisource 4089446 nahwiki 3984588 bswikisource 3879731 amwiki 3774873 azwikisource 3774873 arwiktionary 3670016 ckbwiki 3670016 zhwikinews 3670016 angwiki 3565158 dvwiki 3565158 ganwiki 3565158 mgwiki 3565158 cawiktionary 3460300 ruwikiversity 3460300 eowiktionary 3355443 hifwiki 3355443 hewiktionary 3250585 kshwiki 3250585 eswikiversity 3145728 fiwikibooks 3145728 szlwiki 3145728 wuuwiki 3145728 cewiki 3040870 glwiktionary 3040870 liwiktionary 3040870 lowiktionary 3040870 ruwikinews 2936012 hewikiquote 2831155 rmwiki 2831155 sawikisource 2831155 simplewiktionary 2831155 ugwiki 2831155 vecwikisource 2831155 cowiki 2726297 arwikibooks 2621440 bowiki 2621440 zhwikibooks 2516582 bclwiki 2411724 cswikiversity 2411724 furwiki 2411724 huwikiquote 2411724 idwiktionary 2411724 iswikisource 2411724 kywiki 2411724 map_bmswiki 2411724 stqwiki 2411724 diqwiki 2306867 fiu_vrowiki 2306867 hrwiktionary 2306867 krcwiki 2306867 mkwikisource 2306867 ptwikiversity 2306867 afwiktionary 2202009 bawiki 2202009 extwiki 2202009 hiwiktionary 2202009 iswiktionary 2202009 lawiktionary 2202009 ocwiktionary 2202009 roa_rupwiki 2202009 sawikibooks 2202009 scwiki 2202009 tkwiki 2202009 fawikiquote 2097152 ilowiki 2097152 ladwiki 2097152 mhrwiki 2097152 nrmwiki 2097152 viwikibooks 2097152 bgwikisource 1992294 brwikisource 1992294 cawikibooks 1992294 dsbwiki 1992294 fawiktionary 1992294 pawiki 1992294 sewiki 1992294 sowiki 1992294 thwiktionary 1992294 wawiktionary 1992294 yiwikisource 1992294 zhwikiquote 1992294 cswikibooks 1887436 fawikibooks 1887436 mkwikibooks 1887436 mrjwiki 1887436 orwiki 1887436 bgwikiquote 1782579 emlwiki 1782579 eowikisource 1782579 lijwiki 1782579 mznwiki 1782579 scnwiktionary 1782579 skwikiquote 1782579 trwikiquote 1782579 wowiki 1782579 crhwiki 1677721 euwiktionary 1677721 hsbwiktionary 1677721 kwwiki 1677721 novwiki 1677721 plwikimedia 1677721 svwikibooks 1677721 bjnwiki 1572864 frpwiki 1572864 gnwiki 1572864 kowikibooks 1572864 kvwiki 1572864 miwiki 1572864 nlwikiquote 1572864 nvwiki 1572864 ruewiki 1572864 udmwiki 1572864 brwiktionary 1468006 csbwiki 1468006 cswikiquote 1468006 cywikisource 1468006 lnwiki 1468006 svwikinews 1468006 testwiki 1468006 xalwiki 1468006 bswikiquote 1363148 chrwiki 1363148 jbowiki 1363148 kmwiktionary 1363148 sawiki 1363148 tpiwiki 1363148 trwikibooks 1363148 cbk_zamwiki 1258291 eowikibooks 1258291 fiwikinews 1258291 frrwiki 1258291 idwikibooks 1258291 koiwiki 1258291 mrwikibooks 1258291 nowikibooks 1258291 pcdwiki 1258291 rwwiki 1258291 simplewikiquote 1258291 skwikibooks 1258291 sqwiktionary 1258291 elwikiquote 1153433 fawikinews 1153433 glkwiki 1153433 glwikisource 1153433 hakwiki 1153433 iswikibooks 1153433 mswiktionary 1153433 papwiki 1153433 ukwikimedia 1153433 arwikinews 1048576 azwiktionary 1048576 bugwiki 1048576 dawiktionary 1048576 fywiktionary 1048576 gagwiki 1048576 huwikinews 1048576 iawikibooks 1048576 iewiki 1048576 kbdwiki 1048576 klwiki 1048576 lowiki 1048576 nlwikimedia 1048576 rowikibooks 1048576 rowikinews 1048576 sdwiki 1048576 tawikinews 1048576 ukwikibooks 1016729 kawikibooks 1005363 cswikinews 996352 cvwikibooks 996352 dawikibooks 975667 elwikiversity 962048 swwiktionary 955596 zeawiki 955494 aywiki 954470 bgwikinews 949760 myvwiki 949452 arcwiki 946688 astwiktionary 943718 etwikisource 943718 fiwikiquote 943718 pdcwiki 943718 srwiktionary 943718 slwiktionary 937267 fiwikiversity 933888 vowiktionary 933785 hewikinews 931840 kabwiki 922931 ltwikiquote 922419 kawiktionary 915660 azwikibooks 894873 nlwikinews 894259 cywiktionary 889753 acewiki 889651 hrwikibooks 884224 thwikibooks 878080 kaawiki 869376 glwikibooks 841318 wikimania2007wiki 840192 outreachwiki 832512 jawikiquote 818995 test2wiki 818176 elwikibooks 813772 pflwiki 811315 wikimania2009wiki 805171 wikimania2011wiki 803123 ltwikisource 801382 svwikiquote 788787 jawikiversity 780492 igwiki 776192 ttwikibooks 775680 bhwiki 771379 ukwikinews 757248 cawikinews 754176 abwiki 744140 aswiki 736153 mrwiktionary 732569 skwikisource 725299 mdfwiki 721510 cuwiki 720486 nowikinews 718233 zh_min_nanwiktionary 714547 ltwikibooks 713318 hywiktionary 707174 bgwikibooks 700211 pagwiki 698675 sahwikisource 688640 arwikiquote 687001 slwikiquote 685158 trwikinews 683008 tetwiki 678809 srnwiki 663142 cawikiquote 660582 wikimania2008wiki 646041 towiki 644710 liwikisource 638873 ukwikiquote 621772 nowikiquote 617984 hawwiki 608870 srwikibooks 597299 ltgwiki 594841 rmywiki 590745 pntwiki 583270 lvwiktionary 565248 kowikiquote 565043 wikimania2010wiki 564019 gawiktionary 554496 lbewiki 544870 sawiktionary 516096 elwikinews 514150 sswiki 511180 nawiki 504012 cdowiki 499097 srwikiquote 498380 bmwiki 496025 brwikimedia 492748 tenwiki 489267 kkwiktionary 480665 eowikiquote 475750 avwiki 471654 slwikibooks 471449 eewiki 467046 omwiki 464691 hrwikiquote 464179 hywikiquote 459980 wikimania2006wiki 450867 piwiki 449331 kawikiquote 447488 dawikiquote 441241 kywikibooks 439603 hiwikibooks 439091 rowikiquote 430387 hawiki 429158 svwikiversity 425881 zh_min_nanwikisource 424448 tswiki 421580 viwikiquote 419840 angwiktionary 414310 tawikibooks 413081 idwikiquote 412262 tlwiktionary 411955 smwiki 405401 tywiki 404172 pihwiki 386457 sdwiktionary 382771 bxrwiki 379596 tlwikibooks 374476 mowiki 373555 sqwikinews 367718 simplewikibooks 363520 kgwiki 363315 cowiktionary 352665 iuwiki 350105 yiwiktionary 344985 ttwiktionary 342323 gotwiki 340889 ikwiki 340480 nowikimedia 338329 tkwiktionary 337408 mswikibooks 335667 glwikiquote 332288 newiktionary 329523 sewikimedia 328601 urwiktionary 326860 biwiki 324812 bewiktionary 322048 siwiktionary 320102 usabilitywiki 319078 amwiktionary 318259 skwiktionary 315596 angwikibooks 304025 thwikiquote 294297 mlwikibooks 293273 hiwikiquote 287744 iewiktionary 278630 bnwikibooks 277401 nnwikiquote 276070 etwikiquote 275046 xhwiki 269209 mlwikiquote 266035 bswikinews 264908 lbwiktionary 264806 kswiki 262041 kowikinews 261427 anwiktionary 261324 sdwikinews 261017 wowiktionary 258867 bswiktionary 257843 sgwiki 257024 zawiki 257024 knwikisource 246784 urwikibooks 235929 wikimania2005wiki 234803 kywiktionary 233881 zuwiki 232652 gnwiktionary 227328 etwikibooks 224665 ffwiki 221696 bnwiktionary 214220 nnwiktionary 213606 crwiki 211968 azwikiquote 208793 dzwiki 207974 csbwiktionary 206131 nywiki 205414 thwikinews 205312 cywikiquote 204492 tumwiki 202956 ndswiktionary 198041 eowikinews 195686 stwiki 194867 mkwiktionary 192204 tiwiki 189644 kmwikibooks 188620 iswikiquote 186368 rnwiki 185958 newikibooks 184320 tnwiki 182784 ptwikimedia 182169 vewiki 180224 kuwikiquote 177971 shwiktionary 175206 euwikiquote 174796 lgwiki 169062 chwiki 166297 vowikibooks 160768 fjwiki 154419 klwiktionary 146636 jvwiktionary 146227 uawikimedia 142131 guwiktionary 140800 bewikiquote 132300 snwiki 130457 nahwiktionary 130150 htwikisource 129331 pswikibooks 127590 pswiktionary 125440 ocwikibooks 124108 akwiki 120934 mnwiktionary 120115 mrwikiquote 120012 sqwikiquote 119910 tewikiquote 119296 gdwiktionary 117657 fowiktionary 117555 ugwiktionary 116121 rwwiktionary 111513 twwiki 109260 bswikibooks 107929 tgwiktionary 107008 quwiktionary 104345 hywikibooks 103731 kiwiki 101580 liwikiquote 101273 lawikibooks 100454 brwikiquote 99328 euwikibooks 99225 kuwikibooks 99123 kwwiktionary 97382 afwikiquote 95129 gvwiktionary 93900 tawikiquote 90009 lawikiquote 87449 stwiktionary 85196 mtwiktionary 84889 ruwikimedia 83763 rswikimedia 81510 miwiktionary 80384 tpiwiktionary 79462 ngwiki 79155 chrwiktionary 78233 zuwiktionary 75059 fywikibooks 73932 omwiktionary 73932 bewikibooks 73830 tewikibooks 73113 swwikibooks 70963 iuwiktionary 69939 lvwikibooks 69529 tkwikibooks 67276 sowiktionary 66252 fowikisource 64716 chywiki 63897 tiwiktionary 63692 tgwikibooks 62464 suwiktionary 61235 alswiktionary 60928 cowikimedia 60723 iewikibooks 60723 lnwiktionary 59596 pawiktionary 59596 fiwikimedia 58880 sswiktionary 58265 tswiktionary 58265 cowikibooks 57548 uzwiktionary 57344 nawiktionary 56422 pawikibooks 55398 roa_rupwiktionary 54886 ikwiktionary 54169 mkwikimedia 53964 smwiktionary 53350 hawiktionary 52633 afwikibooks 51916 pa_uswikimedia 51609 chwikimedia 51097 knwikiquote 50688 sgwiktionary 49971 tnwiktionary 46796 guwikiquote 46592 dvwiktionary 46284 kswiktionary 46284 liwikibooks 45568 mowiktionary 45363 fjwiktionary 45056 mgwikibooks 43212 aswiktionary 40448 angwikiquote 38912 trwikimedia 38809 bhwiktionary 38297 abwiktionary 37478 dzwiktionary 37273 orwiktionary 37068 snwiktionary 37068 yowiktionary 36864 biwiktionary 36761 twwiktionary 36761 towiktionary 36249 xhwiktionary 35840 lbwikiquote 35737 bowiktionary 34713 cywikibooks 33894 urwikiquote 33484 aywiktionary 31641 nzwikimedia 28057 uzwikiquote 27545 aawiki 27340 astwikiquote 26931 amwikiquote 26828 suwikibooks 25600 zawiktionary 24985 kywikiquote 23552 advisorywiki 22835 cowikiquote 22630 aawiktionary 21504 astwikibooks 20275 wikimania2012wiki 20275 zh_min_nanwikibooks 19148 iiwiki 18329 zh_min_nanwikiquote 18124 akwikibooks 17920 qualitywiki 17817 jbowiktionary 17408 mhwiki 16384 etwikimedia 14540 chowiki 14028 dkwikimedia 11776 kkwikiquote 11673 kkwikibooks 9420 mnwikibooks 9318 scwiktionary 9113 wowikiquote 8601 knwikibooks 8089 angwikisource 7884 bmwikibooks 7680 ndswikibooks 7680 quwikibooks 7680 nawikibooks 7475 zawikibooks 7270 howiki 7065 xhwikibooks 6860 suwikiquote 6656 ugwikibooks 6553 miwikibooks 6451 kswikibooks 6348 zuwikibooks 6246 guwikibooks 6041 kswikiquote 5836 tkwikiquote 5529 uzwikibooks 5529 chwikibooks 5222 kjwiki 5222 lbwikibooks 5222 lnwikibooks 5222 ugwikiquote 5222 gnwikibooks 5120 gawikibooks 5017 gawikiquote 4915 gotwikibooks 4812 bmwiktionary 4710 hzwiki 4608 nahwikibooks 4505 sewikibooks 4505 bowikibooks 4403 kwwikiquote 4300 rmwikibooks 4300 mhwiktionary 4096 ttwikiquote 4096 alswikibooks 3993 biwikibooks 3993 rmwiktionary 3788 yowikibooks 3788 vowikiquote 3686 avwiktionary 3584 bmwikiquote 3584 aawikibooks 3379 aswikibooks 3276 piwiktionary 3276 rnwiktionary 3276 bawikibooks 3174 crwiktionary 3174 wawikibooks 3072 crwikiquote 2969 mywikibooks 2969 muswiki 2867 nawikiquote 2867 akwiktionary 2764 alswikiquote 2662 krwiki 2560 zawikiquote 2252 aywikibooks 2150 chwiktionary 1945 vewikimedia 1945 quwikiquote 1843 ndswikiquote 1740 krwikiquote 1126
I won't comment much: the biggest wikis are mostly the same (though in a slightly different order), but the Volapük Wikipedia, to continue using this example, is quite lower in the rankings. Actually, the right way to look at this would be to plot the dump size over the number of articles. Here is the result, restricted to the first 80 Wikipedias (not Wikimedia projects) sorted by the number of articles. (If the graph is not correctly displayed below, you can get it here.)
The Volapük Wikipedia is indeed an outlier, with a few other cases.