From Big Dive to Twisted

@giuseppe_futiaBig Dive 2 logo

In June 2013 I attended the Big Dive training program, organized by the Top-IX Consortium, that aims to “boost the technical skills needed to dive into the big data universe and to extract value”.

During the course I learnt: TurboGears, a framework for developing Web applications; MongoDB, a NoSQL data storage solution; the basics of network science and statistics. Also I studied MapReduce techniques and improved my data visualization skills, by learning the D3.js JavaScript library.

This wealth of knowledge has been crucial to realize with Enrico Zimuel the Big Dive final project (entitled "Gramsci devoted"), in which we developed in a few days a prototype of a Web visualization tool to explore the network measurement experiments performed by Neubot.

Here are the photos taken during the course and the final presentation:

Within the Big Dive final project I was initially involved in the front-end development of the Web interface. Once I finished the course, in the second work phase on our prototype I studied Twisted, an “event-driven networking engine written in Python”, and implemented the prototype code to create a REST API module to retrieve Neubot data, which can be exploited by other applications, as well as from our Web interface.

As explained in a previous post, the Big Dive experience led to the development of NeuViz, a prototype data processing and visualization framework to explore network measurement experiments, and to the publication entitled "Visualizing Internet-Measurements Data for Research Purposes: the NeuViz Data Visualization Tool", in occasion of the 2013 AICA Congress.

The slides of the Big Dive final project are available on Slide Share.