The Analytical Center is working on a methodology for assessing state development institutions

27 september 2016

The Analytical Center is analyzing the effectiveness and efficiency of state development institutions. A pilot version of the new methodology for assessing the state development institutions’ effectiveness was presented today at a round table at the Analytical Center. “We need a single methodology to develop an assessment tool because so far we have not had standardized requirements and parameters for assessing the performance of SDIs,” said deputy head of the Analytical Center Vasily Pushkin as he opened the event.

The work is being carried out on the instructions of the President and the government of Russia on the optimization of the development institutions, Tatiana Eferina, the head of the Department for Monitoring of the Analytical Center, noted. “As we were carrying out these instructions we realized that Russia did not have a transparent system for systematically assessing the performance of its development institutions that would be integrated into the system of management decision making,” the expert said. There is a marked mismatch between the priorities of the development institutions and the list of priorities included in the Russian President’s Instruction No PR-2508 dated December 8, 2015 The quality of communications between the development institutions and their recipients remains low. There is significant overlap of functions performed by various development institutions. Their efficiency and effectiveness remain subpar.

“We have looked at assessment methodologies in Russia and abroad, specifically, in Germany, Canada, the USA, France, Sweden, Australia and other countries,” Ms. Eferina said. - “Eventually, we picked the American Program Assessment Rating Tool or PART model as the basis for our pilot methodology.” According to Ms. Eferina, this methodology’s main advantage is that it uses an integrated approach and can be used to easily develop a rating system. It is based around multi-factor analysis comprising 4 modules, each containing between 8 and 12 questions. “It is a well thought-out assessment system with clearly defined procedures. It does away with subjectivism as it is based on a whole series of sources that combine self-reporting and external assessments,” the analyst said. Another important factor is that PART comes as part and parcel of the management decision making system and makes it possible to easily see at which stage the state initiative misfires.

“As a result, we get a quantitative assessment score. And each score is backed with a massive amount of expert work that goes into it: surveys, strategic documents, financial and annual reports and a lot more,” Ms. Eferina summed up. And any development institution is a big project that seeks to solve one specific problem, the expert believes, so it is important to assess not just its economic performance but also other aspects of its activities as well as its overall impact on society, the quality of support it offers, the efficiency of expenses and effectiveness of its operations.

Representatives of the state development institutions supported the initiative to create a single methodology for assessing SDI performance based on PART. Specialists also believe that the methodology should be developed with their participation and must take into account the interests of all participants. “Creating a new PART-based methodology will be useful on two conditions: First, if the assessment is used to make concrete decisions, like whether or not financing should continue. Second, these assessments should not contradict or perform the same functions as other audits,” believes Boris Yaryshevsky, the Strategic Development Director for The Russian Exports Center JSC. The document must also take into account the responsibility and performance of the ministers responsible for specific development institutions.

The pilot ranking of SDIs was carried out on the basis of the information available in open sources as well as the materials SDIs provided to the Analytical Center at the latter’s request. The methodology is currently being considered and discussed with interested parties.