The HEC should never rank Pakistan's universities

Murtaza Haider — Updated May 27, 2015 10:35pm

The HEC rankings reward conformity rather than innovation. —Creative commons

The Higher Education Commission (HEC) in Pakistan should be the last institution to make public its ranking of universities.

The commission is directly responsible for funding research and providing capital for expansion. The university rankings, as a result, are greeted by those at the top. However, those at the bottom would consider the rankings a sign of future denials by HEC to their capital requests.

The recently released rankings list Quaid-e-Azam University (QAU) at the top in the general category. This must have raised the morale at Pakistan's premier research university. Preston University in Kohat, at the other end of the spectrum, was ranked 67, thus ending up at the bottom of the HEC's rankings. The news media highlighted those at the top, but paid scant attention to the state of those at the bottom of the rankings.

Ranking universities is a big business globally that is often led by specialised teams in established publication houses. The Times Higher Education World Reputation Rankings, US News College rankings, QS University rankings, and Maclean's University rankings (Canada) are examples of independently conducted university rankings.

Unlike the HEC, others who rank universities do not have a direct bearing on the universities' future funding and resources.

Being a direct sponsor of universities, the HEC must abandon the practice of publicising its university rankings.

Also read: LUMHS questions HEC ranking of varsities

The HEC rankings are not merely about research excellence or student experience. In fact, the HEC allots a significant weight to criteria that measure universities' compliance to HEC's dictates.

Put simply, HEC rewards conforming universities with a higher placement in the rankings. This is a classic definition of conflict of interest.

Resultantly, one may see obvious omissions or lower rankings of certain private universities who enjoy a much larger national and international repute, LUMS comes to my mind, yet they rank low because of not complying with the HEC's standards.

Ranking regimes are not without bias

The HEC's university rankings are no exception when it comes to bias.

Apart from the obvious conflict of interest, other glaring limitations do exist. For instance, the HEC rankings even consider as input the international rankings of the universities by others. For an independent evaluation, others' perceptions should not matter.

Then, there is the case of double counting.

The HEC favourably considers academic publications in journals of good repute. It then double counts the same by normalising journal publications by the number of fulltime faculty. Later, the rankings double count the number of doctoral students graduated by a university by counting them independently and further reconsidering the same after normalising the graduated doctoral students by the number of fulltime faculty.

Regardless of how one defines the criterion to rank institutions of higher learning, a review of independent rankings will reveal that research-based universities rank higher than the rest. In fact, global rankings of universities often rank Harvard University, Oxford University, Stanford University, and MIT at the top. These institutions also rank the highest in their research productivity.

In my former role of associate dean of research and graduate programs, I was deeply involved in improving and maintaining Ryerson University's MBA rankings. After a careful review of several MBA School rankings, we concluded that business schools with high-impact research ranked at the top.

While other criterion matter, nothing mattered more than research productivity in most business school rankings. This should not come as a surprise. If one takes research out of consideration, there is not much difference left between a university and a high school.

To see if the HEC's rankings gave adequate weight to research, I used the Web of Science index to count the number of research publications produced in 2014 by the top research producing institutions in Pakistan.

The Web of Science index may cover journals in some academic disciplines more than in others. This would naturally bias the outcome. However, as I have mentioned earlier, all rankings are subjective and are subject to bias.

I present in the following table a comparison between the HEC's rankings and the one based entirely on research productivity. One sees significant conformity in the two rankings for the top six research producing institutions. QAU, COMSATS, University of Punjab, and University of Agriculture (Faisalabad) ranked in the top four in both rankings.

It is interesting to note that from a research productivity point of view, King Abdulaziz University is ranked 9 because of co-publications with authors in Pakistan. At the same time, the National Centre for Physics Pakistan is ranked 13 for research productivity, while it is not a degree granting institution.

When I broke down research productivity by research areas, a surprising conclusion emerged. The COMSATS Institute of Information Technology emerged as the leading producer of academic research in social sciences, engineering, and business. While COMSATS does not rank high in the HEC's discipline-specific rankings, it does exceptionally well for research in the same fields.

For health sciences, the Aga Khan University was the obvious research leader, followed by the University of Karachi.

While the HEC rankings list IBA Karachi at the top of the business schools, most reviewing the same would have a tough time swallowing the fact that LUMS did not make it in the HEC's business school rankings. This has more to do with the HEC's subjective criterion that effectively excluded one of the most highly reputed institutions of higher learning from consideration.

I used the Web of Science Index to develop a list of research publication produced in 2013 and 2014 in business and economies produced by authors based in Pakistan.

The results, presented in the following table, put COMSATS at the top followed by LUMS.

PIDE, an established centre of excellence in economics research, and the Lahore School of Economics, an emerging intellectual powerhouse, also made the list of the top business and economics research institutions in Pakistan.

The two leading business schools from Karachi did not fare well for research productivity.

IBA Karachi, an established institution with a great reputation, did not list in the top 20 research producing schools. Also missing was the newly established Karachi School of Business and Leadership (KSBL), which is a joint venture with the Cambridge University.

When I cross-referenced KSBL's results against Google Scholar, I was up for a bigger surprise. KSBL generated only eight mentions in the entire Google Scholar repository.

IBA Karachi and KSBL could have published their research in other avenues that I missed. However, only eight mentions in Google Scholar should be a source of mild concern for the academic leadership at KSBL.

Also read: HEC gives itself clean chit in fake degree scandal

At a time when Pakistan is in the international news for fake degrees and academic fraud, speaking of academic and research quality at home is a welcome distraction. Pakistan's academic scene has the potential to improve if academic institutions become more student- than faculty-centric.

The next time someone ranks universities in Pakistan, they should also consult with the students whose voice continues to be ignored in the operation of universities in Pakistan.