Less time in total, as a bit of the content was skipped. Wikipedia consists of 6,424,367 articles, meaning it should takeĪround two hours to convert all articles to JSON. I kept the defaultįor -w (the number of workers), which is 31. On my machine I got roughly 50,000 articles per minute. Going to use wget -c here, so that we can continue a partial download, just in
View wiki offline download#
Downloading Wikipediaįirst, let’s download the latest XML dump of the English Wikipedia. So today I thought it might be a great day to try a different approach. At some point, it just became too impractical to Pretty good, I ended up with a 250GB database back at the time, which Using uveira, my own command line tool for that. Than the actual dump itself, which at time of writing is 81GB in total sizeĪ year ago I tried this experiment once and used a tool calledĭumpster-dive to load the Wikipedia dump into a MongoDB and access it Workflow – which is terminal-based – and doesn’t end up eating more storage I was looking for a more lightweight approach that integrated well into my Have some pretty terrible prerequisites, like for example Java. Or through other means, but they’re all quite cumbersome to use and in parts
View wiki offline Offline#
That try to offer an offline version of Wikipedia, either based on these dumps Ready-to-use apps (like Kiwix, Minipedia, XOWA, and many more) Unfortunately, these database dumps aren’tĮxactly browsable the way they’re offered by Wikipedia. Unfortunately crawling Wikipedia’s HTML for offline use is not something that’sįeasible - and it’s not even necessary as Wikipedia offers database dumpsįor everyone to download for free.
Some documentation on real life, for which I use Wikipedia most of the time. However, I oftentimes ended up in situations, in which I needed to look up Machine, so that I wasn’t dependent on connectivity to be able to work. Speeds and stable connections (<3 Seoul) or, well, not so much so.īack at the time I took care to have documentation stored offline on my Depending on where in the world I was, I either had superb internet Working stationary usually don’t experience. Travelled quite a lot and hence had to deal with all sorts of things that people As some of you might know, up until the global situation went sideways I
0 kommentar(er)
