Neubot 0.4.10 released

Neubot icon

Version 0.4.10 is the latest generally-available stable release of Neubot, distributed for Ubuntu >= 10.04, MacOSX >= 10.6, Windows >= XP SP3 and FreeBSD, as well as in source format.

Here's what changed in 140 characters: better M-Lab redirection, preparatory work for Win32 auto updates, cleanups, docs and fixes. Install, upgrade instructions at http://www.neubot.org/download, read on for more details.

This release contains 110 commits. The diffstat says: 65 changed files with 1,683 additions and 586 deletions.

Here is the list of what changed and why:

  • M-Lab: use airport names to compile redirection table

    I noticed that it's possible to prepare a much better redirection table for M-Lab, using airport names, embedded in the name of each server, instead of trying to guess where the server is by means of geolocation. It turns out that the old method of generating the redirection table was not bad, and only a few servers were misplaced.

    This is another step towards even better and more robust redirection to M-Lab servers using DONAR, which is a distributed DNS integrated with M-Lab. I am working to adapt Neubot to it, but the work is not ready yet, so, in the meantime, I decided to deploy the airport fix to improve the redirection table a bit.

    Link to commit:

    https://github.com/neubot/neubot/commit/a19f0c3a

  • rendezvous: stop supporting clients without privacy settings

    Since version 0.4.4 (released 25 October 2011), Neubot does not run tests unless the user provides the permission to collect her Internet address. Since version 0.4.6 (released 24 January 2012), Neubot needs the permission to publish her IP address too, in order to use Measurement Lab.

    Still, old clients without any privacy permissions were redirected on the master server. And, similarly, old clients that have not provided the permission to publish were redirected on master as well. I decided to support them, to give users some time to upgrade to newer versions of Neubot.

    Now, it's time to move forward, and, with this change, I am, de facto, removing support for all Neubot users that have not provided any privacy permissions. Judging from the logs, I estimate that I am removing service for 10-20 users, who have not upgraded their Neubot for a long time. I think this is fair, because, since the release of 0.4.4, four months have passed.

    Please, note that in a not-too-distant-future I will also remove support for users that have not provided the permission to publish, for many reasons, including that I want to make master more clever and helpful. That will probably happen when version 0.5.0 will ship.

    I understand that staying up-to-date is annoying, especially for Windows users, and for this reason I am working to finalize the long-awaited auto-updater module for Windows. If everything goes well, experimental support for that should be ready in 0.4.11, stay tuned! :-)

    Link to commit:

    https://github.com/neubot/neubot/commit/6b7f8497

  • Preparatory work for Win32 auto-updates

    As mentioned above, one of the things I'm working on is automatic updates under Windows. Part of the work was to enhance the runner module, which is the Neubot module that runs rendezvous and tests and make sure that only one test at once can run. The enhancements provide the infrastructure to be able to download a file using the runner as it were a test.

    Link to commit:

    https://github.com/neubot/neubot/commit/53d6b43

  • Preparatory work for making rendezvous more smart

    There are some downsides in current rendezvous implementation and, in particular, it takes work to add new fields and provide backward compatibility. I would like, instead, to make the rendezvous more smart and pass back to the client information such as the ISP prefix and name, to allow to e.g. prevent automatic tests to run when you are in a 3G network. To move in this direction I need to remove the obstacles and one obstacle is the XML-based rendezvous. So, this release is the first one to use JSON for rendezvous and more changes are due in the future to refine that and gradually introduce more goodies in the rendezvous process.

    Link to commits:

    https://github.com/neubot/neubot/commit/aa0774fc

    https://github.com/neubot/neubot/commit/f8e02cc1

  • Always use runner_rendezvous for rendezvous

    In Neubot 0.4.6 I introduced runner_rendezvous, a simplified rendezvous client that is run by the runner (the component that runs tests). That was need to automatically and transparently trigger a rendezvous when a user manually runs a test and there are no information on the right server to use.

    Albeit based on the original one, the new simplified rendezvous was simpler and better defined, because I had more clear the problem when I wrote it. So the plan had always been to enhance it and use it to replace (part of) the original rendezvous client.

    With this release that piece of work is finally done and now the old rendezvous client is just a wrapper around the new rendezvous client. This is good news, I have plans to make the rendezvous more smart, as explained above, and having simpler and more rational code helps a lot.

    Link to commit:

    https://github.com/neubot/neubot/commit/0965da40

  • Better MacOS auto-updater

    Working on windows updater, I have borrowed from and improved some pieces of MacOS updater. In this release, I include better code that performs more checks when auto-installing the tarball.

    Link to commit:

    https://github.com/neubot/neubot/commit/8aaf40a0

  • Cleaner bittorrent/peer.py

    Another thing I'm working on is making sure that Neubot can work in 1 Gbit/s and 10 Gbit/s networks. This boils down to reading the code, testing and removing bottlenecks. As usual, re-reading my old code, I've noticed a number of things to be cleaned up, and the set of changes is mature and safe enough to spill in this version of Neubot.

    Link to commit:

    https://github.com/neubot/neubot/commit/3bfdd305

  • Bug fixes

    Since Neubot 0.4.6, I've started collecting know bugs information at:

    https://github.com/neubot/neubot/wiki/errata

    Here is the list of bugs that have been fixed since the previous release, along with the link to the relevant commit:

    • Win32: install in %APPDATA% not %PROFILE%.

      The proper place where to install under Windows is the nonroaming appdata folder (%APPDATA%), not the user home directory (%PROFILE%). Out of the Windows jargon, for me this means that, under Windows 7, Neubot is installed in:

      C:\Users\simone\AppData\Local\Neubot

      Instead of in:

      C:\Users\simone\Neubot

      Fixed by these commits:

      https://github.com/neubot/neubot/commit/29cfe63f

      https://github.com/neubot/neubot/commit/5190ae98

    • MacOS: update installer to privacy policy 2.0.3.

      MacOSX installer privacy policy was at version 2.0 whereas the current version was 2.0.3 (nothing significant changed, nonetheless that must be corrected).

      Fixed by this commit:

      https://github.com/neubot/neubot/commit/0f0a5902

    • MacOS: do not add okfile to generated packages.

      MacOS auto-updater tarballs contained by mistake the ok-file, which indicates that Neubot has been installed successfully. Instead, they should not contain this file, and the autoupdater must add it when a tarball is unpacked successfully.

      Fixed by this commit:

      https://github.com/neubot/neubot/commit/06bba1ac

  • More code rationalizations, cleanups, documentation improvements and regression tests.

Full change log available here:

https://raw.github.com/neubot/neubot/0.4.10/ChangeLog

Diff since previous version here:

https://github.com/neubot/neubot/compare/0.4.9...0.4.10

Commit history here:

https://github.com/neubot/neubot/commits/0.4.10

SHA256sums here:

https://github.com/neubot/neubot/wiki/sha256

Errata here:

https://github.com/neubot/neubot/wiki/errata

Data here:

http://www.neubot.org/data