Searchable BlitzBasic forum archive


Tweet
(Posted 4 years ago) markcwm

Hello all,

about 2 weeks ago I started writing an app to download all forum topics from the BlitzBasic archive mojolabs.nz, this is now done and I won't be sharing the app but I will share the forum files. While this might be copyright infringement I am not making any money out of this so I think I'm safe legally. I split the forums up into languages and you can download them here: bmx-forums, monkey-forums, b3d-forums, bplus-forums, sdk-forums, code-forums and community-forums (Note there are no topics from General Discussion).

I gave up waiting for Google to provide a full index of the BlitzBasic archive (as it may never happen) and found a desktop search engine named Regain that works just like Google once it's indexed the files, which it does really quickly. However, you will need to install Java 1.6+. On MacOS this should already be done, on Windows you will probably want Java SE (Standard Edition) JDK8 from Oracle choosing the relevant architecture (the dev kit has more features than the runtime environment), on Linux there may be a default JRE in the repositories but if not you can manually install it, just get the relevant JDK8 download from Oracle and follow this DigitalOcean how-to for Ubuntu/Debian.

Once you have Java you can set up Regain. For the download get the desktop zip, it seems there is no MacOS version but the Linux version works on Mac. Extract the regain folder to anywhere handy, open a command prompt and "cd" to this folder, to run type "java -jar regain.jar" and it should launch in the taskbar, if it starts indexing you will have to wait a while. Right-click the icon and select Preferences, this opens a page in your default browser, add a directory (it must be a full path) to the forums folder and Save, go to the Status page to know when it's finished indexing, then just go to the Search page. You should get results like this:

105697.html (Relevance: 1%)
GLSL Shaders in BlitzMAX News Forums Code Logs Gallery Specs Search GLSL Shaders in BlitzMAX BlitzMax Forums/BlitzMax Programming/GLSL Shaders in BlitzMAX TomToad (Posted 2016) [#1] Been trying to ... a bit of googling. TomToad (Posted 2016) [#20] Now messing with the vertex shaders. Made a nice
text/html ~/forums/bmx-forums/forums/bmx-programming/105697.html - 49.61 kB - Jan 20, 2018 Cached

That's it, have fun!

(Posted 4 years ago) hardcoal commented:

Thanks Mark!

(Posted 4 years ago) RonTek commented:

This is awesome Mark, thanks! Actually I went ahead and did another slow patient web crawl a few weeks ago using php, but it is not quite complete. This will come in really handy and will soon do a cdn mirror copy just in case.

(Posted 4 years ago) markcwm commented:

Yes, that was another reason why I did this because many other people will do a crawl if nothing searchable exists.

Another thing I should have mentioned is with my copy you can also navigate by the forum pages and if a topic was moved you can navigate to that too and code archive source files are also there. However, you can't navigate to the forums from a topic.

(Posted 4 years ago) RonTek commented:

Awesome, thanks again Mark. This will be a valuable source of info and resources. cheers.

(Posted 8 months ago) RonTek commented:

Mark, just curious if these archives also includes the worklogs. It seems there's a ton of useful snippets and info embedded on most of those logs.

(Posted 8 months ago) markcwm commented:

Hi Ron,

actually no, I never saved the worklogs or general discussion.

mojolabs.nz appears to be down now but you can get it on on Internet Archive:
http://web.archive.org/web/20200416152630/http://mojolabs.nz/logs.php

You can also get them from Blitzbasic.com:
http://web.archive.org/web/20170626134805/http://www.blitzbasic.com/logs/logs.php

I should be able to modify my downloader program to grab them all.

(Posted 8 months ago) RonTek commented:

Thanks Mark, that would be nice.

Yes, but apparently some posts were not scraped by web.archive so still relying on those actual post by now down mojolabs

(Posted 2 days ago) markcwm commented:

Hi Ron,

here is the forum archive edit update, it's not completely finished but I thought I'd better release it now in case I take too long about it. I appear to have forgot to include the b3d sdk in my edits but that's not a big deal is it? The toolbox is finished and the Gallery has a few pages as well as a page of Monkey Apps, all of that is in the toolbox-gallery.zip. Oh and the img folder adds the Blitz icons to the bottom of the forum pages if put at the root path.

I'll leave it to your disgression as to what to do with it!

bmx-forums
monkey-forums
b3d-forums
bplus-forums
code-forums
community-forums
user-worklogs
toolbox-gallery

(Posted 1 day ago) RonTek commented:

This is great! Thanks for the update.

This could take a bit longer, but we'll get there eventually

I appear to have forgot to include the b3d sdk in my edits but that's not a big deal is it?

Yes, not that really but just for posterity.. so no rush.

Reply To Topic (minimum 10 characters)

Please log in to reply