Last week I was presenting a talk at EclipseCON US at Boston US. As usual this is an excellent opportunity to give you an overview of the different trends of the conference.
EclipseCON Boston 2013
TEDx Louvain-la-Neuve
For the first time a TEDx event was hosted in Louvain-la-Neuve last 23 march 2013. It was the 4th anniversary of TEDx and plenty of conferences were organized around the world at this occasion. Like all TED conferences, the spirit of this conference was based on the motto “ideas worth spreading”. Sixteen speakers were invited to give twenty minutes talks on various subjects going from artistic cooking, to collective awareness over politics and bad managements practices in companies. The day was divided in four parts of four talks and between the parts we had the opportunity to discuss with the speakers and the audience on an informal way.
Standards-Based Cloud Service Management
From a Master Thesis to CLOSER 2013
In the scope of a Master Thesis collaboration between the Université de Liège (ULg) – Electrical Engineering and Computer Science Department and EURA NOVA, we have been working last year on cloud standard of service interoperability. This work has been summarized in a paper [1] accepted in the poster session at 3rd International Conference on Cloud Computing and Services Science, CLOSER 2013. The goal of this project was to contribute on solving the lock-in problem inherent to current cloud offerings. We based our solution on open cloud standards, mainly OCCI.
Towards a standards-based cloud service manager
Migrating services to the cloud brings all the benefits of elasticity, scalability and cost-cutting. However, migrating services among different cloud infrastructures or outside of the cloud is not an obvious task. In addition, distributing services among multiple cloud providers, or on a hybrid installation requires a custom implementation effort that must be repeated at each infrastructure change. This situation raises the lock-in problem and discourages cloud adoption. Cloud computing open standards were designed to face this situation and to bring interoperability and portability to cloud environments. However, they target isolated resources, and do not take into account the notion of complete services. In this paper, we introduce an extension to OCCI, a cloud computing open standard, in order to support complete service definition and management automation. We support this proposal with an open-source framework for service management through compliant cloud infrastructures.
Amine Ghrab, Sabri Skhiri, Hervé Kœner, and Guy Ledu, Towards A Standards-Based Cloud Service Manager, proceedings of the 3rd International Conference on Cloud Computing and Services Science, CLOSER 2013, Aachen, Germany, May 2013.
Click here to access the paper.
The captivating AROM of distributed processing
Last month we have had the opportunity to present AROM at the 4th IEEE International Conference on Cloud Computing Technology and Science (CLOUDCOM) in Taipei, the beautiful capital of Taiwan. For the people who are just getting on train, I will quickly here recall what AROM is about, where one can find it and how its taste is.
IEEE CloudCom Conference on Cloud Computing Technology and Science
Last week I was at Tapei with Nam-Luc for presenting the AROM paper. I wanted to come back on the trends of this year at the conference which, by the way, are a really good insight of the hot topics in cloud, distributed computing and HPC. I will not dive into details for each of them, if you have any question just post a comment or send me a mail!
Arom: processing big data with data flow graphs and functional programming
The development in computational processing has driven towards distributed processing frameworks performing tasks in parallel setups. The recent advances in Cloud Computing have widely contributed to this tendency. The MapReduce model proposed by Google is one of the most popular despite the well-known limitations inherent to the model which constrain the types of jobs that can be expressed. On the other hand models based on Data Flow Graphs (DFG) for the processing and the definition of the jobs, while more complex to express, are more general and suitable for a wider range of tasks, including iterative and pipelined tasks. In this paper we present AROM, a framework for large scale distributed processing based on DFG to express the jobs and which uses paradigms from functional programming to define the operators. The former leads to more natural handling of pipelined tasks while the latter enhances genericity and reusability of the operators, as shown by our tests on a parallel and pipelined job performing the calculation of PageRank.
Nam-Luc Tran, Sabri Skhiri, Esteban Zimányi, and Arthur Lesuisse. AROM: Processing Big Data With Data Flow Graphs and Functional Programming, proceedings of the 4th IEEE International Conference on Cloud Computing Technology and Science, IEEE CloudCom 2012. IEEE Computer Society Press, Taipei, Taiwan, December 2012.
Click here to access the paper.
Euranova Academical Research 2012 Vintage
In total this academical year, five Master Theses and one internship have been conducted in collaboration with 3 different academical institutions in Belgium: Université Libre de Bruxelles (ULB), Université de Liège (ULg) and Université Catholique de Louvain (UCL). Let’s here quickly review their goals and contributions.
Data flow graph & stream processing closer than ever
This year we started a research project, named AROM, in collaboration with the Université Libre de Bruxelles. We wanted to evaluate a Data Flow Graph (DFG) processing framework and compare it to a traditional MapReduce (MR) one. In addition, we analyzed high level data manipulation languages such as PigLatin [1] and investigated whether a MR approach introduces an important overhead in the transformation of data operations to the physical execution plan. Going further, we analyzed a typical class of data analytic, the pipelines jobs and we studied the adequacy of DFG for ease of use and performances. Finally, we used functional programming in order to introduce higher order functions to simplify the expression of jobs and to promote the re-usability of operators.
Large Graph Mining
End of July, I was presenting a lecture to the European Business Intelligence Summer School organized by the Univeristé Libre de Bruxelles and the Ecole Centrale Paris. I presented a lecture on large graph mining. In this post I will quickly introduce this fascinating topic.