In the middle of GSoC: The Summer of Sync

It’s been quite a while, truth be told, quite some time without posting updates to this blog.
My GSoC project have been going through a mix being busy, many events happening, trying to find love for new things but I am sure my project is still interesting .
Today I want just to mention the status of my GSoC project ; it’s in the mid-term evaluation and we expect Google paying for all ‘well-going’ projects .

I have a detailed wiki page about my project : OpenMRS Sync, Create a new Sync Node .Everyone is welcome to leave useful comments on this project. The source code is available under the terms of OpenMRS Public Licence and can be browsed from OpenMRS website as well as they can be ‘checked-out'(downloaded) via subversion with the command

svn checkout http://svn.openmrs.org/openmrs/branches/data-synchronization-admin-ui


Other things actually are happening in my coding environment:I have been undergoing series of switching Operating Systems between Windows, Linux and Solaris and I have some insight about what does things in the best way; I have been trying to find love for Groovy/Grails at the same time I am trying the Qt toolkit .Groovy seems to be a powerfull Jav-based agile framework, easy to configure and deploy. Touching it is like drinking milk from your grandma, and when you get satisfied you feel and find that you have more energy . Qt is a cross-platform desktop GUI framework using C++and acquired by Nokia last year, Qt is the parent of KDE Desktop environment for Linux.Trying Qt is like eating bread from your granddad tagged with honey inside; you get stronger and healthier. The reality behind things like these(trying new things) should be :”Always, there is a place where you want to go, there is a person who you want to be with, there is something you always want to see happenning, there a level you always wish to be on, the is a goal you always want to achieve ! The list doesn’t end and when you feel you don’t have neither one of these desire, you don’t enjoy your life .” .

I the past 40 days, I have seen many other things happening around us and among them as I am writing: ATRACO club from Rwanda won the East-African soccer cup(CECAFA), Rwanda has named other new born gorillas (Kwitizina<www.kwitizina.org>: this is really a new brand for the Gorilla tourism in Rwanda, and it demonstrates the efforts deployed in both tourism promotion and conservation ) , Rwanda has celebrated 15 years after National Liberation and the end of Tutsis Jenocide <www.rwanda15.org>. An other event was the departurte of MJJ , the King of Pop, to the land of peace, and I can’t say anything more personal than “In paradisum deducant te Angeli; in tuo adventu suscipiant te martyres, et perducant te in civitatem sanctam Ierusalem. Chorus angelorum te suscipiat, et cum Lazaro quondam paupere æternam habeas requiem.“.

This is not the end, please stay turned to my blog and I love to hear your comments .

svn checkout http://svn.openmrs.org/openmrs/branches/data-synchronization-admin-ui

Last week OpenMRS GSoC activities

Last week was really busy forme, I was packaging everything for traveling back to Kigali . At least now, I get time to write a couple of blog posts . I am seating in WA Dulles airport,  waiting for the plane that take me to Brussels; I am waiting  almost for the whole day and I can now summarize what I’ve done on OpenMRS during last week .

My GSoC project is again about Sync: “Data synchronization is a new OpenMRS feature allowing synchronization of data amongst a set of loosely networked servers. Such ability to exchange data is essential for operation of EMR system in rural areas where connectivity amongst sites maybe unreliable yet the need for timely centralized collection and analysis of data from remote sites exists.”

The synchronization feature was designed in a parent-child hierarchical model, to allow data to flow from from remote sites to more central parent and vice versa . A parent OpenMRS server can have many child servers and child can have only one parent . At synchronization, a child server send new change-sets to its model (SYNC REQUEST) and receives all new change-sets from the server (SYNC RESPONSE) . This can be done either via the web or with a disk drive (file) that can be carried over the parent while the child is off-line and  the parent will also issue an off-line sync response to carry back to the child . The sync via file doesn’t mean that you have to move physically to the place where the the parent server is installed ; OpenMRS is a web app and any where you can access the parent app you can sync your off-line child !

There are two additional changes on the OpenMRS data model that synchronization does :

1) synchronization_* tables are added to the data model for storing sync settings & configuration and they also store the temporary sync import/export records .

2) GUID indexes : to ensure data exchange between different OpenMRS systems, the ID fields are not enough to identify a record because there are from different MySql installations, so the GUID index columns are added to all data that can be sync-ed .

My actual project’s main aim is to provide an automated way of creating a new sync node(i.e.child) and provide the appropriate user interface . This was done in manual process and required much administrative knowledge. When creating a new sync node(i.e.child) you had to:

• register newly created child with parent
• back up parent server DB and move the backup to the new child server
• restore parent’s DB
• assign new server sync ID
• change any server identifying information from parent to child (i.e. form entry server URL)
• test sync connection between parent and child and finally establish periodic sync schedule

In order to achieve these project I have to start from the actual sync code . As last year I was commiting to the synchronization-admin-ui branch, I think I am going to commit to it even for this project . So I started by resolving few issues in the actual code and I merged the synchronization_bidirectional branch to the sync-admin-ui branch so that all sync changes after last GSoC be available to the sync-admin-ui . This was not a simple task because the sync_bidirectional branch also merges from trunk .

I just used Subclipse merge feature and I let both the old and new versions of code be there then I removed the old revisions where it was necessary and, sometimes I had to use a piece of regex in order to be fast .

For example, Subclipse should form somewhere two blocks of code with different versions and limit them with

<<<<<<< .working
//Code
=======
// Code
>>>>>>> .merge-right.r7385

Then in that case I used the following regex and replaced it with an empty space in order to keep the merger-right version of the code

(<<<<<<< \.working([\x00-\xFE]*?)=======)|(>>>>>>> \.merge-right\.r7385)

This is how I merged and it’s working perfectly .

My next step now is to find out a way of cloning the MySql database from the parent DB and apply it to the new child instllation automatically . If you have more ideas about how to achieve this, please leave your comments .

Seattle Theaters

This was really amazing, I spent the last two months doing an internship with RealNetworks, Inc.

I was assigned to a challenging software engineering project and everything went well . My supervisor was always complaining because the internship was so short, and 6 months should be the best . Anyway my project was so successful and it attracted a lot of people in  my team .

Apart of the work, I get to know a lot of people and Seattle especially . This was my first trip to the US , the country where everything is possible !

Sometimes I got to hang out with friends and got to watch movies in the theaters around . Well, this was a very adductive fun and it’s the reason why the title of this post is called so . For my colleagues who like coding, I have a fun of code that always came out after watching one of the SciFi series . Guess which :

try{
energize();
}catch(UnknownUniverseException ex){
ex.printStarTrek();
}
Isn’t that fun ? The really came out once I wanted to write ex.printStackTrace and I wrote ex.printStarTrek instead .

Here is alist of the movies I enjoyed while in Seattle :

1- Fringe : a kind of new X-Files ; you know what ,  I already saw someone in Olivia’s face here in Seattle and I am applying for a Computational Bio-Chemical Engineering internship position with Massive Dinamic , WHAT CAN’T WE DO ?  (  — This is a joke )

2-Star Trek : This was awesome and I enjoyed it even before I knew it was so famous (at the time when they released the new season).

3-Angels & Demons : Oh my goodness , what a Camerlengo ? I was happy to see the movie starting in the LHC near Switzerland but I didn’t imagine it’s gonna be a Da Vinci Code of some sort . Anyway I am not a part of Illuminati ! And at the end of the movie the guy(Camarlengo) got into trouble , and I remembered my old high school latin text about Catilina: Quo usque tandem abutere, Catilina, patientia nostra?

and

4- Drag me to Hell was the Killer ! Don’t even dream of the story because it can be so harmful .
That was my fun time, tell me what you think about these movies .

DWR/AJAX/JavaScript Hacks: The secrets of JavaScript in the adress bar

A couple of weeks ago a friend of mine send me a JavaScript code that I called “Balancez-Balancez” ;  it’s really fun to play with it . Just with your browser go to any website (e.g. www.baidu.com ), then  after the page is loaded copy the following code and paste it into the address bar of your browser :
javascript:R=0;x1=0.1;y1=0.05;x2=0.25;y2=0.24;x3=1.6;y3=0.24;x4=300;y4=200;x5=300;y5=200;DI=document.links;DIL=DI.length;A=function(){for(i=0;i-DIL;i++){DI[i].style.position='absolute';DI[i].style.left=Math.sin(R*x1+i*x2+x3)*x4+x5;DI[i].style.top=Math.cos(R*y1+i*y2+y3)*y4+y5}R++;};setInterval('A()',5);void(0);

What happens: all the links in the website become a beautiful CHINESE DRAGON DANCING !

The principle of this code is collecting some HTML elements(i.e. links) and animate them continuously (changing their style positions) along an elliptical line .

It was really pretty looking and it conducted me to dig deeply and find out how to go beyond  this practice of JavaScript in the address bar for just manipulating the HTML page : I thought about AJAX .

In fact if the page has an AJAX port, meaning the back end can handle some XMLHttpRequest, you can  build an out-of-page JavaScript code that runs into the address bar of the page and calls some functions in the back-end of your application .

Example: Use DWR

Consider you have exported a DWR function that gets the app logs from the server . You can just omit to put the JavaScript imports in your page header and load them dynamically with a JavaScript in the address bar ; you can also override the Ajax Callback function, you can override the callback handler and so on . Here is the example :


javascript:
script=document.createElement('script');
script.type='text/javascript';
script.src='/js/page.js';
document.getElementsByTagName('head')[0].appendChild(script);
css=document.createElement('link');css.type='text/css';
css.rel='stylesheet';
css.href='/css/page.css';
document.getElementsByTagName('head')[0].appendChild(css);
scriptA=document.createElement('script');
scriptA.type='text/javascript';
scriptA.src='/dwr/util.js';
document.getElementsByTagName('head')[0].appendChild(scriptA);
scriptB=document.createElement('script');
scriptB.type='text/javascript';
scriptB.src='/dwr/engine.js';
document.getElementsByTagName('head')[0].appendChild(scriptB);
dwr={};
dwr.engine={};
DWREngine=dwr.engine;
DWRLogsService={};
DWRLogsService._path='/dwr';
DWRLogsService.giveMeTheLogs=function(callback){
dwr.engine._execute(DWRLogsService._path,'DWRLogsService','giveMeTheLogs',callback);
};
if(DWRLogsService==null)alert('DWRLogsService=null');
void(0);
logsHandler=function(logsList){
newTable='<table cellpadding="4" cellspacing="0" width="100%" border="0">';
for(i=0;i<logsList.length;i++){
newTable+='<tr><td>'+logsList[i]+'</td></tr>';
}
newTable+='</table>';
document.getElementById('logsDiv').InnerHtml=newTable;
};
displayLogs=function(){
DWRLogsService.giveMeTheLogs(logsHandler);
};
setInterval('displayLogs();',1000);
void(0);

Since this technique is not very fair, there are three main points you have to consider when making building the Javascript code for the address bar:

1)Avoid new line (‘\n’ or ‘\r’) or space into your code : we are in the browser address bar !

2)Don’t use the name  ‘var’ for declaring your local or global variables ; this should cause a new space into your code.

4)Call void(0) at the end of your code .

I hope this can help anyone who want to try and please leave your feedback and lets share the experience .

Pushing it forward, The Summer of Love !

A couple of weeks ago, I definitely started using Ubuntu “officially”  on my new laptop, and I realized that it’s really SUPER-COOL . This is one of the most successful Desktop Linux distros .The reasons are so many : I was obviously tired of windows and most of my colleague coders are using Unix-based O.S. This was also one of my great goals of 2009 and it may be a reason to say “GanBei” in Chinese for those who knows . I will no more rely on Cygwin or MinGwin for coding C/C++ or Qt with Eclipse .

The are really big efforts deployed in open-source world to produce for free those SUPER-COOL tools like Linux or other tools we use in our everyday life or others used by institutions and organizations . Open-source is a kind of “green business” whose model is not understood by many people, vulnerable to failure but very remarkable for their success . There isn’t a single development on this world which is not aware of this kind of revolution . It should be the change we wish to see !

One of the greatest programs in contribution to open-sourrce is the  Google Summer of Code(TM) ,“Since its inception in 2005, the program has brought together nearly 2500 successful student participants and 2500 mentors from 98 countries worldwide , all for the love of open source coding – Google “ . I am happy to be accepted again for the 2009 version of GSoC . I will be working with OpenMRS an open source medical record platform for developing countries . I will be working on the Data Synchronization feature of OpenMRS and my primary mentor Maros is also a kind of super-cool ; I met him last year in Kigali .

OpenMRS has been deployed in different african countries especially as EMR system for fighting against AIDS, Malaria and Tuberculosis , ” Our world continues to be ravaged by a pandemic of epic proportions, as over 40 million people are infected with or dying from HIV/AIDS — most (up to 95%) in developing countries. Prevention and treatment of HIV/AIDS on this scale requires efficient information management, which is critical as HIV/AIDS care must increasingly be entrusted to less skilled providers. Whether for lack of time, developers, or money, most HIV/AIDS programs in developing countries manage their information with simple spreadsheets or small, poorly designed databases…if anything at all. To help them, we need to find a way not only to improve management tools, but also to reduce unnecessary, duplicative efforts. As a response to these challenges, Open Medical Record System (OpenMRS^®) formed in 2004 as a open source medical record system framework for developing countries — a tide which rises all ships. OpenMRS is a multi-institution, nonprofit collaborative led by Regenstrief Institute, Inc. (http://regenstrief.org), a world-renowned leader in medical informatics research, and Partners In Health (http://pih.org), a Boston-based philanthropic organization with a focus on improving the lives of underprivileged people worldwide through health care service and advocacy.     – OpenMRS “

There are many special things in this Summer of Code : I have a neighbour summer of code student, Mugisha in Kampala (Makerere University student), who is working with the same OpernMRS project, so we should even have our meet-up in East Africa . This is the fifth anniversary of Google Summer of Code, I hope we will make it the best summer of code, yet the Summer of Love, love for coding . I love coding on my Ubuntu Laptop !

Mozilla Firefox Properties editing

I was seeking a way to change some Mozilla Firefox configurations , and it’s not easy to find all configurations and options under the Menu Tools->Options. I just wanted to enable print without the Print Dialog Box and print within a pop-up . Someone helped me to get this tips which I think are very important for someone who want to print within a web-application (i.e. reports, receipts, …) .

Configuring Mozilla Firefox

1- Type about:config in the address bar of the browser and hit ENTER

2-Confirm the warning message

3-To Disable the Print Dialog Box :

a- right-click on page > select New > Boolean

b- Preference name : print.always_print_silent

Boolean value : true

c- search for print.show_print_progress

double-click to change its value, set it to false

4- To Disable Pop-up blocker:

search for browser.popups.showpopupblocker , set value to false

I think this can help to any one who seeks it like me .

GSOC 2008: OpenMRS Sync maintenance tasks

As suggested by Gsoc organizers today is the *suggested* pencil-down date. In my new updates I included the Synchronization maintenance tasks as the last part of the project .It’s divided into three parts : Sync search , sync journal and imports archiving and managing some sync timeouts from the sync admin ui.
Now you can search for sync records according to their display name and you can view or edit them from the results list obtained.
For the journal archiving task , I created a new class that can zip up the contents of a folder into a .zip file using Adler32 checksum . The class org.openmrs.serialization.ZipPackage
compress the sync journal or import up to 30% . For example the contents of %APPDATA%\OpenMRS\journal are archived into %APPDATA%\OpenMRS\archive\journal\journal_2008_08_09_22_26_00_288.zip and the contents of the original folder are cleaned up .

The screen shot of the maintenance page is presented here :

As the last step of this project I am going to check everything in the codes like the log messages,result messages from Spring,to check if there is anything missing or any misunderstanding message, to add comments for developers guidance ;I have also to update the sync administrator helping screen and refine the project wiki page for future reference.

Thanks

GSOC OPENMRS Project news:Editing sync record items payload

These days I am implementing the final part of my soc project . One of the features in the last commit was the ability of the Sync Administrator to edit the Sync record items payload .To that end I created a method in the DWRSynchronizationService.java that gets the sync Item content and then sent it to the web client .

public String getSyncItemContent(String guid, String key) {
    String content = "";
    Collection<SyncItem> itemList;
    if (guid != null && guid != "" && key != null && key != "") {
        itemList = Context.getSynchronizationService().getSyncRecord(guid).getItems();
        for (SyncItem item : itemList) {
            if (item.getKey().getKeyValue().equals(key))
            content = item.getContent();
        }
    }
    return content;
}

After creating that function, I have to configure it in dwr.xml

<create creator="new" javascript="DWRSynchronizationService">
<param name="class" value="org.openmrs.web.dwr.DWRSynchronizationService" />
<include method="testConnection"/>
<include method="syncToParent"/>
<include method="getSyncItemContent"/>
<include method="setSyncItemContent"/>
</create>

The content String got in synchronizationViewRecordList.jsp (via Ajax) is just in mxl tags format and it looks like :

<org.openmrs.ConceptName><guid type="string">f6b29ecf-b15e-102b-ac59-58f4c1e745a2</guid>
<concept type="org.openmrs.Concept">f5aa80e9-b15e-102b-ac59-58f4c1e745a2</concept>
<description type="string">edited A disease caused by a lack of vitamin C and characterized by
 spongy gums, loosening of the teeth, and bleeding into the skin and mucous membranes.</description>
<locale type="locale">en</locale><name type="string">VITAMIN C DEFICIENCY</name>
<dateCreated type="timestamp">2004-06-01T00:00:00.000+0200</dateCreated><shortName type="string"></shortName>
<creator type="org.openmrs.User">fc193cd8-b15e-102b-ac59-58f4c1e745a2</creator></org.openmrs.ConceptName>

It was quite useful to have that payload in a single text field on the Admin UI so that the Sync Admin can edit it like that ,but it could result in a sync failure if the admin alters the tags structure or changes some crucial data like the GUIDs . So we decided to parse it with javaScript and display the payload values in different text fields where the Admin can edit and save them . To save that content we have to rebuild it from html tags .

The parsing code is here:

  function showContentResult(result) {
    DWRUtil.setValue("loadContent", "");
    domParser = new DOMParser();
    xmlDocument = domParser.parseFromString(result,'application/xml');
    root=xmlDocument.documentElement;
    rootElt=root.nodeName;
    cn=root.childNodes;
    newHTML="<form id='contentForm' action='' method=''><table width='100%' border='0' cellspacing='0' cellpadding='0'>";
    for(i=0;i<cn.length;i++){
      newHTML+="<tr class='syncTr' ";
      if(i%2==0)newHTML+="bgcolor='#C4D9D9'";
      else newHTML+="bgcolor='#E7EFEF'";
      newHTML+="><td><strong>"+cn[i].nodeName+"</strong></td><td>"+cn[i].attributes[0].value+"</td><td   height='30'>
<input id='field'"+i+" type='text' value='";
      if(cn[i].childNodes.length==1)
      newHTML+=cn[i].childNodes[0].nodeValue;
      newHTML+="' size='50' /></td></tr>";
    }
    newHTML+="<tr><td>&nbsp;</td><td>&nbsp;</td><td   height='35' align='left' valign='bottom'><input type='button' name='saveButton'
   value='  Save  ' onclick='setItemContent();'/></td></tr>";
    newHTML+="</table></form>";
    document.getElementById("contents").innerHTML = newHTML;

  }
  function getItemContent() {
    DWRUtil.setValue("loadContent","Loading Item payload ...");
    DWRSynchronizationService.getSyncItemContent(currentGuid, currentKey,showContentResult);

  }

And the screen shot is here:

An other task implemented was to perform standard search for sync records.This is actually implemented according to the record name.

In the next step , I have to finalize all maintenance tasks and update the help screen .I hope that will be the end of the soc , and our codes should get in production quickly .

GSOC MIDTERM REVIEW AND OTHER UPDATES

So far my project was reviewed and so far you come up to get my updates.Yes, last Thursday I get in the OpenMRS developers conference call.I was a little bit late and hopefully my mentor had come in advance and he had talked about my project and the work I already achieved at the time.This summer of code was not exactly a Summer holiday like some other students. In fact I am doing it in parallel with the school classes and other small assignments.

Back to the code review , I am unable to explain the feelings i had at the review time. My skype connection was so bad at campus (where I live) so late I decided to go to an internet cafe for better connection ; it was not the best idea to attend a conference in a public place like that but it was my last chance to get my project reviewed .Really I got a little bit frustrated for making a public speech online to some *software experts* located in different places of the globe.The content of my review was just a summary of what has been made to the Sync Admin UI and the next trip in the advance of the project.The developers suggested everything concerning the project have to be fixed before the end of the GSOC so that we should come up with something really productive.

As a description of my midterm goals you can refer to the wiki page of the project as well as in the last updates of this blog . For this next term of the project I have to Implement mainly the Sync maintenance tasks as well as refining the tasks useful to the pages achieved in the last part of the project . To that end, I ended up by bringing up the Sync Item payload to the Sync Record View/Edit screen via DWR/Ajax .
Right now it’s in text format with tags but you can edit and save it ;that seems even useful for the Sync administrator at the time .The  problem with this is a possible data corruption if the administrator try to edit some sensitive information like IDs,GUIDs,or tag names or if he alters the tag structure in the payload.So I will have to parse the payload as long as  it’s loaded from DWR then put it in some text fields where the user have option to edit only the text tag content.To save the edited payload, I will  have to rebuild the content from the fields and save them via the same channel (DWR/Ajax).

An other thing that needs to be fixed is to update of the project wiki page for future public reference ; hopefully I will come up with everything working properly and my updates are going to turn to regular.

GSOC NEW UPDATES

It’s now time for midterm evaluation. The time has been running so quick so that I couldn’t imagine how it ends up to this.I have almost achieved a part  of the work expected  and the project screen shots are available on the project wiki page. I am planning for a conference call with the OpenMRS core developers for code review.I think they should give me more ideas about the next step project orientation if I pass the midterm evaluation . The current source codes have been updated with small css changes on the user interface . Hope I will pass the evaluation , and you should still updated for the next project development.

– Antoine