Many new services-based applications transcend the bounds of a single organization, and data definitions often loom as the most pressing challenge when integrating these extended systems. This was the case for the California Independent System Operator (ISO), a not-for-profit corporation that manages the state's market-based wholesale power grid.
The independent California grid power broker embarked on the road to SOA around 2004. The immediate job was to better connect the varied information systems of power market participants.
"There were a number of systems that had kind of kluged together very disparate or very common services that would be better abstracted as a single system," said Jami Long, manager of business solutions. "We were also largely lacking a common business vocabulary—industry wide and even within the ISO itself." Changes in data handling were required.
The changes affected more than 100 power vendors. To get all these disparate systems to communicate effectively, the California ISO (CAISO) used a common information model (CIM). But this was no simple task.
Typically, a common information model contains a specification and a schema. The schema contains model descriptions while the specification contains integration details. These objects and relationships provide common definitions of management information for an architecture that can then be extended for use by third-parties. Groups like the Distributed Management Task Force (DMTF) and the Object Management Group (OMG) have worked long and hard to help businesses standardize such models. Industry-specific CIMs include an International Electrotechnical Commission (IEC) CIM fashioned to the needs of the electrical industries.
Close to 100 power companies participate on the California ISO's grid, each of which needs to retrieve information from the central system. Not only did these vendors have their own infrastructures, they did not share a standard information model, Long said.
As the data architect of this SOA implementation, Long said she wanted the ISO to create an information model that would set a standard for its market players. She also wanted to avoid any tie in to a vendor-specific technology that might cost its trading partners in licensing.
SOA implementation simplifies compatibility
The ensuing Market Redesign and Technology Upgrade project took about five years and involved a great deal of custom internal development, although a standards-based commercial ESB was employed. The organization runs on Java and PL/SQL and, through the redesign, adopted WC3 standards and now uses SOAP with its Web services. The SOA implementation remained technology-agnostic, Long said.
The new infrastructure allowed CAISO to split its three primary power grid zones into more than 3,000 nodes, each reflecting local power generation and delivery costs. The system sends status reports on all of these nodes every five minutes. Web services of vendors that provide required data need only to conform to the CIM, said Long.
These transactions can get big. Long said large message handling required a great deal of work. Another challenge was to make all this data available in as close to real time as possible. She said her team decided on an approach of using attachments for large messages and, in some cases, breaking down very large transactions into smaller pieces.
Another major challenge was Web service versioning. Market participants were not required to upgrade their systems every time the California ISO updated its system. As a result, older versions of many Web services need to remain operational.
But even these challenges were no match for that of defining a common information model, Long said.
Semantics require precision and new thinking
"We have about 100 market participants right now, ranging from very large utilities down to relatively small companies that want to participate in our market but don't have the same level of resources available," said Steven Berberich, CFO and vice president of technology and corporate services at CAISO. "Trying to make this work for all levels of business partners was difficult."
For example, defining the word "bid" took four hours alone because the term had different systemic and semantic qualities from business to business. Unlike long established sectors like the financial industry, Long said energy markets are relatively new. Within them, you won't find the same well-defined, long established terminologies. This makes it both important and challenging to create a common information model.
Under the new information model, the CAISO has created the business vocabulary for the rest of its market to conform to.
The system went live last April and the need for frequent testing and adjustments made it quite expensive, Long said, but the benefits have been speed and adaptability. The system is much more effective at predicting market congestion and accurately representing the markets through pricing, said Berberich. The system can even look ahead and predict on an hourly timetable what power prices will be the next day. Berberich said this faster, more distributed system has lead to healthy competition and a better value for power.