During the week of May 1-4, I attended CondorWeek 2012 in Madison, WI. The event was hosted in the Wisconsin Institutes for Discovery at the University of Wisconsin-Madison.
The first day consisted of tutorials explaining the basics of Condor usage, workflows, administration, and security. Having spent more time doing general support with OSG and working with the support workflow, I haven't spent as much time as I'd like with Condor, but the tutorials did show me some things that I would like to try. I think that next year, these tutorials would be even more useful for me after trying some new things with Condor in the coming months.
The final three days consisted of researchers, admins, and companies showing and discussing off the multitude of ways that they use Condor to enhance and extend their workflows. Of particular interest to me was the way that DreamWorks used Condor to send jobs to their rendering software to create their animated films. They showed the trailer for Madagascar 3, which is their first movie to be created using Condor from beginning to end in the rendering process.
It was interesting to hear other outfits explain how they have had to work around some things Condor currently can't do and see how the Condor team was curious to see if these things could be implemented into coming versions of Condor. The team was very interested to expand and enhance Condor, which is already a very capable product.
Overall it was a good experience that made me aware of all the things that can be done with Condor. I look forward to trying some of these things for my own, and hopefully returning next year with a much better understanding of Condor as a whole so that I can gain even more information from the talks and tutorials.
Monday, May 7, 2012
Thursday, March 29, 2012
RabbitMQ & CometD
We've been experimenting with following new services in the last few month here at GOC.
- event.grid.iu.edu (RabbitMQ/AMQP Server)
- comet.grid.iu.edu (CometD Server)
RabbitMQ (event.grid.iu.edu) allows GOC, and OSG users to publish and/or subscribe to various messages generated by our services, and OSG in general. Currently, we receive following messages.
- RSV status changes
- OIM updates
- GOC Ticket updates
- GIP information changes (prototype)
Anyone can subscribe to these messages in XML format, and be notified in real time using AMQP messaging client. APIs are available in many languages including, Java, PHP, Python, etc..
CometD (comet.grid.iu.edu) allows us to push messages to our various web applications. For example, GOC Ticket uses it to display users who are currently viewing a ticket. If someone updates a ticket while someone else is viewing, it will send page refresh request to all viewers. CometD can also be used to implement features such as chat, shared editing, and other functionalists.
CometD itself is a Java application framework where we can implement various services that client (web browsers) can make requests to. CometD acts as a glue between RabbitMQ and the web browsers. For example, "GOC event service" in comet.grid.iu.edu subscribes to RSV, OIM, and GOC tickets events, and pools all recent events. A web browsers can then make a request to download these events during the initial loading of a page, and it will subscribe to "new event" queue on comet in order to receive new events in real time until user closes the page.
By using Event & Comet services, we can implement interesting features such as Realtime GOC event (prototype) in MyOSG. My current goal is to continue experimenting with RabbitMQ/CometD and see what I can (and can not) accomplishing using these tools.
If anyone has an idea about what we can do with these tools, please feel free to send me a message.
Wednesday, February 15, 2012
New Home for OSG Web Site
On Frebruary 28th the OSG Webpages located at www.opensciencegrid.org will move to the OSG Twiki. This move corresponds with the upcoming conclusion of a contract with the Chicago based web hosting service Tilted Planet.
During the scheduled production service update on the 28th, browsers will be redirected from the current location to the new location on the main OSG Twiki page. A mockup of the new page can be seen at twiki-itb.grid.iu.edu.
This is the first step, and likely an interim web page home, in a project that will affect the OSG Public Web Pages, the OSG Twiki, the DocDB, and possibly other OSG services with web-UI’s. Evaluation of content management systems, wikis, and documentation file database solutions has already begun and will continue over the next several months. Please contact the GOC (goc@opensciencegrid.org) if you have a suggestion for packages you think should be evaluated.
Monday, November 21, 2011
GOC holiday schedule
From 24/Nov through 27/Nov the GOC will be operating on a Holiday
schedule. Staff will be available to respond to emergencies but
routine operations will resume at start of business Monday 28/Nov.
The GOC wishes its users and OSG staff a happy and satisfying
Thanksgiving Holiday.
schedule. Staff will be available to respond to emergencies but
routine operations will resume at start of business Monday 28/Nov.
The GOC wishes its users and OSG staff a happy and satisfying
Thanksgiving Holiday.
Thursday, November 3, 2011
Moving Services to Bloomington
As you know, the GOC updates services on the second and fourth Tuesday of each month.
The update scheduled for November 8th marks a milestone for the infrastructure team.
After this date all GOC services (with one exception) be be hosted exclusively in
the Bloomington, Indiana data center.
Previously, most services had two instances, one physically hosted in Indianapolis
the other in Bloomington. These instances are in DNS round robin allowing users
of these services transparent use of either instance. The GOC will continue to
operate (at least) two instances and keep them in round robin, but both instances
will be in Bloomington.
So why the change? Originally, the Bloomington machine room was extremely unreliable.
Problems included a leaky roof, insufficient cooling and power and a lack
of space. In short, the systems hosted there had outgrown the facility. The machine
room in Indianapolis was larger, newer and considered more reliable. The old Bloomington
machine room went down during a thunderstorm when it was discovered that both electrical
feeds were, at one point, hung from the same utility pole. (Care to guess where the
lightning struck?) Two weeks were required to restore power during which many of the
university enterprise services were unavailable. This situation was clearly unacceptable
so the university decided to invest $37.2M in a new, state-of-the-art data center.
The 92,000 sq. ft. Bloomington data center is designed to withstand category 5 tornadoes.
The facility is secured with card-key access and 7 x 24 x 365 video surveillance.
Only staff with systems or network administration privileges have access to the machine room
requiring biometric identity verification. Fire suppression is provided by a double interlock
system accompanied by a Very Early Smoke Detection Apparatus (VESDA). Three circuits feed
the Data Center, traveling redundant physical paths from two different substations.
Any two circuits can fully power the building. A flywheel motor/generator set conditions
the power and provides protection against transient events and uninterruptible power
supplies protect against failures of moderate (~1 hour) duration. Dual diesel generators
can provide power for 24 hours in the event of a longer term power failure. In house
chillers provide cooling. Externally supplied chilled water plus city water can be used
in the event of a failure of this system.
Several advantages are realized by hosting all instances in one location. Service failures
associated with the network between Indianapolis and Bloomington are avoided. By using the
same LAN, DNS round robin can be replaced with Linux Virtual Server (LVS) giving control
of round robin to the GOC rather than the DNS administrators at Indiana University. Also
avoided are failures associated with the loss of one of two data centers. It is trivial to
move virtual machines from host to host since the IP address of the VM does not change, a
property allowing detailed load balancing on all VM hosts.
The GOC looks forward to continuing providing services with the availability OSG users
have come to expect.
The update scheduled for November 8th marks a milestone for the infrastructure team.
After this date all GOC services (with one exception) be be hosted exclusively in
the Bloomington, Indiana data center.
Previously, most services had two instances, one physically hosted in Indianapolis
the other in Bloomington. These instances are in DNS round robin allowing users
of these services transparent use of either instance. The GOC will continue to
operate (at least) two instances and keep them in round robin, but both instances
will be in Bloomington.
So why the change? Originally, the Bloomington machine room was extremely unreliable.
Problems included a leaky roof, insufficient cooling and power and a lack
of space. In short, the systems hosted there had outgrown the facility. The machine
room in Indianapolis was larger, newer and considered more reliable. The old Bloomington
machine room went down during a thunderstorm when it was discovered that both electrical
feeds were, at one point, hung from the same utility pole. (Care to guess where the
lightning struck?) Two weeks were required to restore power during which many of the
university enterprise services were unavailable. This situation was clearly unacceptable
so the university decided to invest $37.2M in a new, state-of-the-art data center.
The 92,000 sq. ft. Bloomington data center is designed to withstand category 5 tornadoes.
The facility is secured with card-key access and 7 x 24 x 365 video surveillance.
Only staff with systems or network administration privileges have access to the machine room
requiring biometric identity verification. Fire suppression is provided by a double interlock
system accompanied by a Very Early Smoke Detection Apparatus (VESDA). Three circuits feed
the Data Center, traveling redundant physical paths from two different substations.
Any two circuits can fully power the building. A flywheel motor/generator set conditions
the power and provides protection against transient events and uninterruptible power
supplies protect against failures of moderate (~1 hour) duration. Dual diesel generators
can provide power for 24 hours in the event of a longer term power failure. In house
chillers provide cooling. Externally supplied chilled water plus city water can be used
in the event of a failure of this system.
Several advantages are realized by hosting all instances in one location. Service failures
associated with the network between Indianapolis and Bloomington are avoided. By using the
same LAN, DNS round robin can be replaced with Linux Virtual Server (LVS) giving control
of round robin to the GOC rather than the DNS administrators at Indiana University. Also
avoided are failures associated with the loss of one of two data centers. It is trivial to
move virtual machines from host to host since the IP address of the VM does not change, a
property allowing detailed load balancing on all VM hosts.
The GOC looks forward to continuing providing services with the availability OSG users
have come to expect.
Tuesday, November 1, 2011
EGI Technical Forum
Wrote this for the OSG Newsletter, but thought it would be good to drop here also.
Henry Kissinger famously asked, “Who do I call if I want to call Europe?” I was reminded of this quote when I attended the EGI Technical forum in Lyon, France and then visited CERN in September. While I don’t need to call Europe as a whole, as Operations Coordinator for OSG I may need to contact any one of the 30+ National Grid Infrastructures (NGI) that make up the European Grid Infrastructure (EGI) in times of an operational crisis.
Accompanied by Scott Teige the OSG Operations Technical Lead, we presented material on both technical and personal communications between OSG, WLCG, and EGI including Global Grid User Support System (GGUS) ticket synchronization, availability and reliability reporting, Berkeley Database Information Index (BDII) information exchange, and various other ways to keep communication channels open between OSG and our European counterparts. We came away with several technical action items from the SAM team. In addition, OSG Operations took a seat as a non-voting member of the EGI Operations Management Board. We are also participating in the WLCG Operations Technical Evaluation Group which will provide input to chart the future activities within the WLCG. We look forward to continued collaboration with WLCG and EGI on an operational level.
~ Rob Quick
Henry Kissinger famously asked, “Who do I call if I want to call Europe?” I was reminded of this quote when I attended the EGI Technical forum in Lyon, France and then visited CERN in September. While I don’t need to call Europe as a whole, as Operations Coordinator for OSG I may need to contact any one of the 30+ National Grid Infrastructures (NGI) that make up the European Grid Infrastructure (EGI) in times of an operational crisis.
Accompanied by Scott Teige the OSG Operations Technical Lead, we presented material on both technical and personal communications between OSG, WLCG, and EGI including Global Grid User Support System (GGUS) ticket synchronization, availability and reliability reporting, Berkeley Database Information Index (BDII) information exchange, and various other ways to keep communication channels open between OSG and our European counterparts. We came away with several technical action items from the SAM team. In addition, OSG Operations took a seat as a non-voting member of the EGI Operations Management Board. We are also participating in the WLCG Operations Technical Evaluation Group which will provide input to chart the future activities within the WLCG. We look forward to continued collaboration with WLCG and EGI on an operational level.
~ Rob Quick
Sunday, October 2, 2011
Rob and Scott go to CERN and EGI Technical Forum
Early in the afternoon of 15th of September Rob Quick and I set off for Geneva to visit our collaborators at CERN and attend the European Grid Initiative (EGI) technical forum. We arrived without difficulty at 7:55 AM on the 16th and decided to get the paperwork needed to become CERN users completed. Paperwork being what it is, this turned out to consume the entire day. Many thanks are due to David Collados who picked us up and at days end delivered us to our hotel.
We used Saturday to adjust our clocks by six timezones, working the entire day before actually helped. We hopped a bus to Ferney just across the border in France. There was an open air market where I bought some olives, bread, beer and goat cheese. Turns out David lives near the market and he and his wife gave us a ride back to Geneva. We visited the UN and gorged ourselves of fondue at a small local restaurant. Later in the evening Rob and I made dinner of my Ferney purchase.
On Sunday we hopped a train to Lyon, France for the EGI conference. Quite an enjoyable trip through the mountains. Our talks were on Monday and we learned some interesting things and met many people we knew only by e-mail or con-call. Wednesday was a dinner at a Paul Bocuse restaurant featuring a dessert that made quite the impression on both of us. Raspberries, Chocolate Mousse and Ice Cream. On Friday we knocked off early and did tourist things by visiting the Roman ruins near the confluence of the Saone and Rhone rivers. Those folks built to last. Saturday was back on the train to Geneva.
On Sunday we visited the old part of Geneva. As far as I could tell, old was defined as being contained within the original city walls. I ate a huge bucket of Mussels cooked in white wine, garlic and saffron. Rob had perch and we both had fries, bread and beer. Three days of very productive meetings with our CERN counterparts then back home Thursday at noon. We had a bit of a delay between Washington and Indianapolis but nothing serious.
We both needed Friday to recover, Saturday I started catching up on chores neglected in my absence. Monday I will do the same at work. All in all, a productive if tiring trip.
S
We used Saturday to adjust our clocks by six timezones, working the entire day before actually helped. We hopped a bus to Ferney just across the border in France. There was an open air market where I bought some olives, bread, beer and goat cheese. Turns out David lives near the market and he and his wife gave us a ride back to Geneva. We visited the UN and gorged ourselves of fondue at a small local restaurant. Later in the evening Rob and I made dinner of my Ferney purchase.
On Sunday we hopped a train to Lyon, France for the EGI conference. Quite an enjoyable trip through the mountains. Our talks were on Monday and we learned some interesting things and met many people we knew only by e-mail or con-call. Wednesday was a dinner at a Paul Bocuse restaurant featuring a dessert that made quite the impression on both of us. Raspberries, Chocolate Mousse and Ice Cream. On Friday we knocked off early and did tourist things by visiting the Roman ruins near the confluence of the Saone and Rhone rivers. Those folks built to last. Saturday was back on the train to Geneva.
On Sunday we visited the old part of Geneva. As far as I could tell, old was defined as being contained within the original city walls. I ate a huge bucket of Mussels cooked in white wine, garlic and saffron. Rob had perch and we both had fries, bread and beer. Three days of very productive meetings with our CERN counterparts then back home Thursday at noon. We had a bit of a delay between Washington and Indianapolis but nothing serious.
We both needed Friday to recover, Saturday I started catching up on chores neglected in my absence. Monday I will do the same at work. All in all, a productive if tiring trip.
S
Subscribe to:
Posts (Atom)