Although the project has been a success there were a number of issues that made the project more difficult than originally perceived. These can be broken down into the following broad categories: HCI techniques, personal time and technical.
HCI Techniques: Although we did have a HCI expert on the advisory side for this project this person was overloaded with other tasks and was only able to provide minimal input into the project. This meant that certain techniques such as personas; Focus groups or task-based evaluation was only discovered at a late stage in the project.
Personal time: It turned out to be far harder than expected to obtain time with researchers using e-SC than expected. The original plan had been to hold a workshop in which users could provide input as to how we could improve the SDK. However, as it was almost impossible to get all people together in the same place at the same time we eventually had to change this to one-on-one discussions with users of different abilities. In future work we would seek to hold one-to-one interviews to develop personas rather than attempt a full workshop.
Technical: A number of technical issues caused problems during the project. The variation between Web Browsers plagued us during the project meaning that significant amounts of time were used to ensure that layouts and styles worked across platforms. On completion the wizard tool dropping the user into the expert view at the end, which was not the original intention. Users, however, saw this as a benefit, and as such we decided to keep this as part of the design.
In general we feel that this project not only allowed us to develop a better SDK for e-SC but also helped us to discover the merits and techniques required for improving usability and learnability. Something that we will now be applying over the rest of e-SC.
The project has been a great success based on the original goals of replacing the original SDK with a new integrated online tool as part of the rest of e-SC. This we feel has increased the usability of the SDK significantly. Before this work the development of new blocks for e-SC normally required intervention from the original development team to set up the SDK for the user and then work with them for the development of new blocks. Changes to the core of e-SC and network issues regularly broke the SDK and required further intervention, which often decreased the chances that users would pursue the use of e-SC.
In terms of the learnability of the new SDK, the introduction of the wizard support has greatly reduced the barrier to entry for this work. Allowing users to progress with less need to have significant support from the e-SC development team.
As the development of this new SDK was such a step-change from the original SDK it was difficult to derive quantitative metrics to mark its success. Such metrics could only be defined in terms of the number of users who develop their own blocks or the reduction in help requests for block development. In the short scale of the project this was not possible to observe, especially as the new SDK was only available towards the end of the project. However, more qualitative results have been possible by demonstrating the new SDK to users of the original SDK and those whom we surveyed originally about the SDK. The feedback about the new SDK was very positive.
The intention of this project was to remove the main drawback with e-Science Central (e-SC) – the fact that the Software Development Kit for integrating new workflow blocks into the system was still an independent stand-alone application with a completely different user interface to the rest of e-SC. The rest of e-SC is provided as an online (Web 2.0) style application in which the user requires nothing locally installed other than a Web browser. This was in contrast to the SDK which presented a different Java based GUI containing large amounts of superfluous information and options (only applicable to the developers of the SDK) such as deep hierarchical tree structures for where elements needed to be placed.
To use the SDK created a steep learning curve for any potential user, as they would first need to install a number of software dependencies, including Java and Maven, before they could install the SDK. This was further complicated as updates to the core e-SC software base would often require equivalent updates to the SDK. Communication between the SDK and the core e-SC often failed due to networking problems and proxy issues. We were also aware that a number of users preferred to develop their code in their own IDE before importing it into e-SC as a workflow block.
The new SDK has been developed as part of the online e-SC Web based application. This has immediately removed the need for local software installation, software versioning issues and network issues. The presentation style of the new SDK is now more in fitting with the style of the rest of the e-SC application. All of these help reduce the learning curve for a user who wishes to move from developing workflows with pre-defined blocks to developing their own blocks.
The SDK now supports two usage modes those of expert and wizard support. The expert user interface allows access to all of the features that a user needs to have access to in order to import pre-developed code as a block within e-SC. The wizard support interface guides a user through a set of screens allowing a less experienced programmer to install a new workflow block within e-SC. On completion of the wizard the user is dropped into the expert mode where they can either save the new block or tweak the block before saving. Although this was not the original intention, though after showing the new SDK to the existing user base it was decided to keep this as it was felt that it allowed more of a transition from using the wizard to using the expert mode.
Documentation for using the new SDK has been developed though as the coding of the new SDK took more effort than expected the integration of the help information into the SDK – providing interactive help – was not completed during the project. This will be added at a later date.