“If people in universities, who have the rare ability to convert research into products are not rewarded, and their careers are stifled, they will leave the country and go to the developed world where such abilities are very highly valued and rewarded. As a matter of fact this has already been happening for a number of years, and the country has suffered badly because of it.”

In our view, professors or students who have the ability to convert research into products are rarely, if ever, rewarded by universities anywhere in the world. Rather, it is the marketplace that rewards them: Bill Gates and Steve Jobs were college drop-outs who did not make their mark in the cocoon of a university fellowship program. Rather their ideas and innovation took off in the competitive environment of the open market.

Thus, while we agree that entrepreneurial individuals are sorely needed in the country to transfer the benefits of research and intellectual output at universities, we contend that it is not the mandate of the Higher Education to focus on incentivizing them directly.

HEC is not the panacea. It is one government agency, with limited clout and a shrinking budget. HEC’s focus must remain on allowing our universities to hire and retain the best and brightest researchers and educators that are available, and giving them the opportunity to reach their full potential. That in itself is a formidable task, and executing it well requires making difficult choices. HEC does need to “return to Pakistan” and focus on areas of research and inquiry that are more suitable for Pakistani researchers given our limited resources and our unique developmental needs. To this end, HEC can nudge researchers into areas that are most relevant to Pakistani context.

Government agencies and organizations, like the Ministry of Science and Technology and Pakistan Software Export Board, as well as public-private R&D funds, like the National ICT R&D Fund, need to play the leading role in commercializing research coming out of the universities. Organizations like the National ICT R&D Fund not only have the necessary funding base but their very structure as a public-private partnership makes them ideally suited to carry out this risky but essential purpose.

HoD Ahmad rightly points out that we need a sustained effort to invigorate the industrial base and subsequently create employment. Yet, it’s not the job of university professors; it will be an error to evaluate their worth from a task that is not theirs. Instead, alternative avenues should be provided to support people who have the “rare ability to convert research into products” to thrive and to do what they do best, while not distracting HEC from it core and vital purpose.

Throughout my life I have hated mathematics with a passion. I hated its rules and notations. I hated the fact that I had absolutely no say in whatever was going on in the class. I just had to sit there and listen to my math teacher go on and on about formulas, notations needed to write these formulas, practice questions which would help us memorize these formulas and eventually “practical problems” which were supposed to exhibit the relevance of these formulas in everyday life although even the eight year-old me could tell that these were merely the same practice questions loosely disguised in the most unlikely of social situations known to man. And frankly, I didn’t care. I didn’t care where x was, or how much older Mary was than her brother Mark or when train A would reach London. As far as I was concerned math was an obsolete science to which I didn’t want to contribute to and which, for the most part, didn’t really want me to contribute to it anyway.

Therefore it comes as a surprise to many people that I am currently a Computer Science major focusing on theoretical computer science, which is basically a branch of mathematics. I, who had once famously given a speech to my seventh-grade math class about the pointlessness of mathematics, am now the one trying to explain to other people the beauty of Erdos’ brilliant proofs. And it all started with the following beautiful proof of the infinity of prime numbers:

For any finite set  {p_1,p₂…p_r} of primes consider the number n= p_1..p_2..p₃…p_r +1. This n has a prime divisor p but this is not one of the {p_1,p₂…p_r}, otherwise p would be a divisor of n and the product  p_1..p_2..p₃…p_{r ,} and thus also of the difference n-( p_1..p_2..p₃…p_r) =1, which is impossible.  So a finite set {p_1,p₂…p_r} cannot be the collection of all prime numbers.

I first heard of this proof in the first lecture of a discrete mathematics course I took during my sophomore year at university. The instructor didn’t even write the proof down, with all its messy set notation. He just told us about the idea of putting the prime numbers together in a group and showed us what goes wrong if we assume the group to be finite. At first I thought this was one of those introductory shenanigans professors deploy in the first class to get students interested. How could something so simple be counted as math? Where were the fancy symbols and the list of variables with their definitions? Where was the list of steps used to reach the conclusion? Where were the ten similar questions I needed to solve at home for practice? This was simply a clever idea used to solve a problem. Surely, this couldn’t be math! But, as I have learnt in the past year, this is basically what math is: a set of simple ideas used to solve problems. Sometimes the problems can be simplified to older problems for which people have already come up with solutions. Sometimes ideas which have been used to solve a certain problem can be used to solve an unrelated problem. But the simplicity of the process remains intact. It is the ‘idea’ which is at the heart of all mathematics, and to come up with ideas you just need creativity (and maybe a pencil and a notebook).

If a course can change the path of a person’s life, then this discrete math course changed mine. In the course of nine weeks, I was introduced to the kind of math I hadn’t even known existed. For the first time in my life I didn’t feel like a robot while doing math. I actually had to think about the problems and figure out strategies for solving them. While I was introduced to techniques like induction and graph theory, for the most part my assignments and exams required me to come up with my own strategies based on these techniques and my own logical arguments and common sense. Math was like an elaborate game and finally I felt like it actually wanted me to take part.

So, this brings us to the central question: why did I, and countless other students, hate elementary and high school math? What needs to be done to make mathematics more interesting to students? Although I do not have any experience teaching mathematics, I do remember the reasons why I hated it so much and know exactly what eventually made me realize that I wanted to study a branch of mathematics as my major. For the sake of this article, I am going to ignore factors which affect all subjects alike and focus on why math has become such a hated subject.

Looking back at my years of struggling with high school math the first word that comes to mind is boredom. And this was not caused by a lack of interest in school because I was generally a very enthusiastic kid. I loved studying languages, history, and science. It was just math that I dreaded. And looking back at the way math is taught it comes as no surprise. While all other subjects are taught as an amalgamation of the history, foundations, rules and applications of the subject, math is mainly limited to the rules of the subject. Take a typical sixth grade science class. I remember learning about the effect of different factors on the rate of evaporation by placing different shaped beakers filled with water all over the school campus. What followed was a memorable class in which we all had mock “evaporation races” as we timed the beakers to see which one would lose its water first.It was only once we had made our own conclusions about which factors affected evaporation, that our teacher explained Brownian motion to us. She also mentioned factors such as surface area and wind-speed, which most of us had been able  to conclude for ourselves based on the observations we had made.

Now compare this to a typical sixth grade math class. Looking back, sixth grade was when some of the most wonderful mathematical concepts were introduced to us. It was in the sixth grade that we first encountered the idea of a variable and started to really analyze shapes. Statistics was introduced, and we started manipulating probabilities to get results which even now give me the feeling of being able to predict the future. But in the midst of all these amazing ideas, this is how a typical math class would go:

Teacher: An isosceles triangle is a triangle which has two sides of equal length. Okay?

Students: YES!

Teacher: So what is an isosceles triangle?

Students: A TRIANGLE WHICH HAS TWO SIDES OF EQUAL LENGTH !

And you can bet one of the questions on the progress test would be: “What is an isosceles triangle?”. In such a situation who would be interested in math? And these are not just two extreme examples I have mentioned to prove my point. Science that year continued to keep us hooked: we grew plants in inky water, caught insects in jars, experimented with mirrors and discovered the material we were supposed to learn, while in math we moved on to triangles which had no sides of equal length (I honestly don’t remember what they were called, though I think it begins with an s) and other lexical atrocities.

You may argue that science is an extreme example and that math just doesn’t have the exciting material needed to keep students hooked. While science teachers can use models, take their students outside or perform simple experiments to demonstrate their material, math teachers have nothing to interest a group of thirty kids. Not only do I disagree with this, I actually claim that it is the other way round and that it is the math teachers that have it good. While science teachers need extensive (and often non-available) funding to buy lab equipment and take their students out on field trips, all a math teacher needs are thirty pencils and notebooks. And how does he keep them interested? Well, he actually asks them to do some math. Do you remember the puzzle we probably all tried as kids in which we had to draw a house without lifting our pencils. That is just a simple example of a Eulerian path. And those complicated strategies for winning card games that our older siblings tried to explain to us were mostly simple applications of probability. The tower of rings of increasingly small diameters which we had to shift to another peg is the most common example given for recursive algorithms. The list of interesting mathematical problems which we solved willingly as kids is endless. Nim, Hex, magic tricks, and riddles in which we had to find loopholes in logical arguments are all example of the math we enjoyed as children and it is these problems which should be bought to the classroom to make math classes more interesting.

Another issue which I find with the way mathematics is taught, which is closely related to the first, is the extreme and almost exclusive emphasis on the utterly mundane aspects of mathematics. Take the isosceles triangle example above. Would it really have mattered if we had called the triangles, “triangles with two equal sides”? Maybe shortened to TWTES (pronounced tevtes). What’s important are the properties of these triangles. Instead of asking a child to spend time trying to memorize the pronunciation and spelling of this weird word, she should be asked to think about how they are made, and how the angles inside this triangle are related to each other. I am pretty sure if a child made a dozen different TWTES’ she would figure out most of their  properties for herself and she would actually enjoy the mental excursion of discovering these properties instead of hastily be given a list of them in the last fifteen minutes of class.

Admittedly, there are some terms and jargon that a student of mathematics must learn in order for the classes to be held smoothly and for the students to eventually take part in the wider mathematical discourse. But no other subject puts even half of the emphasis that math places on its lexicon. Take the example of chemistry. If a subject has the right to focus on terminology it is chemistry, with it’s multitude of  symbols, chemical formulas and specific reactions. But not once do I remember a chemistry teacher reciting the names of the elements along with their atomic symbols. Instead, we focused on the elements and their reactions and any time we needed help deciphering a symbol we could simply look it up on the huge periodic table taped to the classroom wall. Maybe that is what mathematics needs: a periodic table of shapes and functions which would be taped to the wall of every classroom. Then, children all over the world could forget about mathematical terminology and actually do some math.

And by ‘doing math’ I don’t mean the mindless repetition, or solving exercise problems at the end of every chapter. As a result of school mathematics, most people end up believing math is the application of known rules to problems that we know the rules can solve. That is the job of an accountant or a cashier or an insurance planner. A mathematicians  job is much simpler. He must come up with the rules that other people are to use. When faced with a problem, he is not told that it can be solved using the second trigonometric identity; that is what he must figure out. And while this is harder than simply applying a set of rules, the result of coming up with a solution is infinitely more rewarding. You can compare the two as the difference between the joy a child feels in having an adult place him on a bike and push him along, and the joy he feels when he races through the park himself. It is hard to teach him how to ride and it might take him ages to learn but all parents understand that the end result is worth it. Math teachers should definitely do the same with their students.

And if difficulty was such a major barrier, why doesn’t it stop teachers of other subjects from trying to get their students to appreciate the beauty of their fields? By the end of high school most of us have faced the toughest aspects of most of the other subjects. We have read Iqbal’s poetry and critiqued it with our peers. We have a deep understanding of how the major systems of the body work. We have built electrical devices and have made original pieces of art in a range of different mediums. Then, why is it that most of us only experience the joy of coming up with a true mathematical proof well into our undergraduate programs? Surely there is something wrong going on here.

To seed this discussion, we present commentary from Dr. Pervez Hoodbhoy and Dr. Atta-ur-Rehman. Dr. Hoodbhoy presents his opposing point of view, arguing that the measures presented in the article were inadequate, and further that the conclusions drawn from the metrics were flawed. Dr. Atta-ur-Rehman, founding (and former) chairman of the HEC, who led the higher education reform effort during his tenure, responds by pointing to data that, in his view, shows the depth and breadth of the reform’s success.

We invite our readers to contribute their thoughts on what metrics are appropriate for measuring the success of higher education within the context of Pakistan.

NOTE: Both commentators have significantly shaped the landscape of Pakistani education over the last few decades. We request our discussants to avoid personalizing the discussion and to maintain a civil and constructive tone.

SYNOPSIS (We will continue to compile the synopsis of the discussion as it progresses: Last Update 8:02am EDT, September 22nd, 2009.)

The metrics suggested, thus far:

1. QUALITY OF RESEARCH (Editors, Zeeshan Khan): Citation and variants on their measures like the h-index, are standard measures of research quality on an individual, institutional, and national level. Number of patents is another measure, though .
2. QUANTITY OF RESEARCH (Atta-ur-Rahman): This measures gross research activity.
3. QUALITY OF TEACHING (Pervez Hoodbhoy): Metric?
4. QUALITY OF UNIVERSITY GRADUATES (Pervez Hoodbhoy, Fakhruddin Habiby): Surveying employers or assessing performance in international tests.
5. ACADEMIC FREEDOM (Pervez Hoodbhoy): Metric?
6. ACCESS TO UNIVERSITY FACILITIES (Atta-ur-Rahman, Khurram Shafique): Libraries, laboratories, internet connectivity, communication facilities, sports facilities,
7. EVALUATION BY NEUTRAL EXPERTS (Atta-ur-Rahman): Survey of a group of neutral experts, like the World Bank, USAID, etc.
8. UNIVERSITY ENROLLMENT (Atta-ur-Rahman): The increase in university-going adults can be measured by census.
9. UNIVERSITY-INDUSTRY LINKAGE (Fakhruddin Habiby, Anwar): The number of industry supported projects which were initiated with University-Industry partnerships and their effectiveness based on industry feedback.
10. LOCAL RELEVANCE OF RESEARCH (Editors): Metric?
11. QUALITY OF CURRENT STUDENTS (Pervez Hoodbhoy, Khurram Shafique):  Performance in standardized tests conducted every year, performance in local and international competitions such as Mathematics Olympiads and Programming Contests.

Notes:

(Pervez Hoodbhoy) Self-citations are a serious problem when using citations as a metric for measuring quality of research reform. [paraphrased]

(Abdullah Sadiq) While strengthening the research effort in the universities is important, the most urgent need is to concentrate on producing quality teachers for the lower tears of education. [paraphrased]

(Khurram Shafique) A pedestrian publication in the field of networking or multimedia is likely to receive more citations than a good publication in a less explored field in mathematics, say, non-standard analysis.

(Fakhruddin Habiby) another ‘tool’ that is used to push the citation number higher is formation of ‘citation-coalition’ within research groups.

(Omar Javed) Categorization of universities into subsets, and adoption of relevant performance criteria for each subset. Three fundamental questions: what is taught (Undergraduate and Graduate Instructional Program classifications), who are the students (Enrollment Profile and Undergraduate Profile), and what is the setting (Size & Setting)”

(Shafiqur Rehman) …the success or falure or HE reforms must only be judged by opinion of the common stake-holders (students, teachers and administration) of the public sector universities.

(Affan): we need to tweak ratings/rankings such that we are able to measure any progress happening in Pakistan, progress small enough that it is not lost by existing metrics.

There are four aspects of the comments of Dr. Pervez Hoodbhoy that need to be considered:

1. Firstly, Dr. Hoodbhoy himself admits that there has been a huge increase in international publications at QAU after HEC came into existence when he mentions the number of international publications in the two time periods. Strangely, he picks a six year period, 1998-2003, and then compares it with the subsequent 4.5 years (?) , 2004 to mid 2008, (the correspondence occurred in August 2008, so he could not possibly have had access to the figures for the entire year) I can only assume that he has mentioned 2003 by mistake in the second “5 year” period as there is no reason to include the publications of the year 2003 in both time periods, which he has done. It is clearly unfair to take two time periods of different durations and compare them.
2. In the first 6 year period (1998-2003), Dr. Hoodbhoy admits that there were only 631 research publications from QAU, but in the second 4.5 year period these had risen to 1482 research publications, a tripling of publications on average per year, even by his own estimates.
3. As the HEC programs began in 2003 and their real impact occurred 2-3 years later, a year-wise comparison is far more relevant than an average over a 5 year period as the dramatic change that has occurred gets partly masked when a 5 or 6 year average is taken, though it is still very visible. Dr. Hoodbhoy ignores the figures that Dr. S.T.K. Naim had worked out that in the year 2004, there were only 84 research publications from QAU (an average of only 7 publications per month), but by 2008 they had increased many fold.
4. The citations argument used by Dr. Hoodbhoy is invalid as citations increase with the passage of time. Dr. Hoodbhoy, therefore, wrongly compares the citations of papers of an earlier period with those of a later period. To clarify this issue further, if two papers of equal quality and in a similar field are published, say in 1998 and 2007, and the citations of both are counted in 2008, then the paper which was published in 1998 will have accumulated more citations by 2008 because of the much longer 10 year time period, than the paper published in 2007, as that would have had only one year for the citations to accumulate. Dr. Hoodbhoy is therefore comparing apples with oranges when he tries to compare citations of papers published in an earlier period with a later time period. In order to fairly compare citations, the same duration of time period must be taken. Thus if one takes 1998 publications and counts the citations till 2008, then one will need to take the 2008 publications and count their citations till the year 2018, before one can compare the figures for the citations of the two sets fairly.

The undeniable fact is that the total number of research publications from universities in Pakistan was only about 600 per year till 2001 but then started rising rapidly, and by the year 2008 it had increased to over 4,300! Brazil achieved such an increase over a 35 year period between 1960 to 1995, which Pakistan achieved in only 6 years. After my appointment in March 2000 as the Federal Minister for Science and Technology in Pakistan, I convinced the government to enhance the budget for science and technology in Pakistan by 6000% between July 2000 to October 2002. After my appointment as Chairman, Higher Education Commission (Federal Minister) the budget for higher education was similarly increased by 2400% during 2003 to 2008. Major achievements during these periods were:

1. Establishing 51 new Universities and awarding institutions during 2002-2008,
2. Tripling university enrollment (which had reached only 135,000 from 1947 to 2003) to about 400,000 in 2008,
3. Establishing a powerful Digital Library which provides free nation-wide access to every student in every public sector university to 45,000 textbooks/research monographs from 220 international publishers as well as to 25,000 international research journals,
4. Establishing video-conferencing facilities in most public sector universities that allow lectures to be delivered live and interactively to students in Pakistan from technologically advanced countries
5. Enhancing salaries of academics so that salaries of University Professors were increased to a level about five times the salaries of Federal Ministers, with a corresponding reduction in tax from 35% to only 5%, in order to attract the brightest young men and women into academia,
6. Promoting research through a massive research grant program which resulted in a 600% increase in ISI abstracted publications from about 600 per year in 2001 to 4300 research publications in 2008, accompanied by about 1000% increase in international citations in the same period,
7. Placing a satellite in space (Paksat-1) which is now used for distance learning by the Virtual University,
8. Establishing video-conferencing facilities in most public sector universities and initiating a lectureship program, allowing live interactive lectures to be delivered from technologically advanced countries,
9. Providing free access to scientists/engineers anywhere in the country to sophisticated instruments installed in any institute in Pakistan.

The Bottom Line: In the final analysis, it is not what I or Dr. Hoodbhoy think about the developments, but what is the opinion of neutral international experts who have carried out detailed year-long reviews of the developments during the period that I was heading the Higher Education Commission. A few extracts are given below:

1. Prof. Fred Hayward (independent international educational consultant from USA) carried out a detailed analysis of the developments and published an article entitled “Higher Education Transformation in Pakistan: Political & Economic Instability,” Date: Number 54, winter 2009 Source: International Higher Education Quarterly. I quote: “The news about Pakistan over the last few years has been dominated by reports of political turmoil, terrorism, religious fundamentalism, economic decline, and the Afghan War. What has been missed is the phenomenal transformation in higher education over the last six years, which represents a critical development for Pakistan and a potential engine for growth and national recovery.”
2. Report of US-AID about HEC states that “We are very impressed with the breadth, scope, and depth of the reforms implemented by the HEC since 2002.  No other developing country we know has made such spectacular progress.”
3. World Bank Report is very complimentary of many excellent programmes introduced.
4. British Council: The report states: “I have worked in many countries in South America, the Middle East, North Africa, and in Russia and India, over the last six years.  None in my view, with the exception of India, has the potential of Pakistan for the UK university sector, largely because of the dynamic, strategic leadership of the Chairman of HEC”.
5. Nature: Several articles and editorials have appeared in the world’s leading science journal “Nature”  (the most recent in the issue published on 3rd September 2009) in which the very significant progress made by Pakistan in the higher education sector has been applauded and the need for the new government to built on the solid foundation laid has been stressed.
6. Science Watch (Thomson Reuters) has ranked Pakistan as a rising star in five disciplines, more than in any other country of the world.

Join the Discussion!
What are the correct metrics to measure higher education reform in Pakistan?

Related Posts:

1. Nature’s Coverage of Higher Education Reform in Pakistan: A Response from Prof. Pervez Hoodbhoy
2. Nature’s Coverage of Higher Education Reform in Pakistan: A Conversation with Athar Osama
3. Pakistan’s Higher Education Funding Holds Many Lessons for Developing Nations: Nature

Editors Note: Dr. Atta-ur-Rahman is a leading scientist and scholar in the field of organic chemistry from Pakistan. He has served as the Federal Minister for Science and Technology, the Federal Minister/Chairman of the Higher Education Commission, Adviser to the Prime Minister on Science and Technology, and the President of the Pakistan Academy of Sciences. Dr. Atta-ur-Rahman is credited by many for reviving the higher education and research practices in Pakistan. The views expressed in this article are solely those of the author and do not necessarily reflect the views of STEP.

Unfortunately, I find the response as unsatisfying as the original article. Since Nature is unwilling to accord me a chance for a satisfactory reply on its pages, I shall clarify the basis of my criticism in some detail here.

In the said article, strong conclusions have been derived from weak data. The authors have not dared to ask the basic questions whose answers are essential for ascertaining whether there has been actual progress in Pakistan’s higher education system and, if so, by how much. Instead, in giving a thumbs-up, numbers have been quoted that have doubtful significance. Take, for instance, the claim that:

“In mathematics, for example, an average paper by a Pakistani author is cited around 20% more than the worldwide average for the discipline”.

Before returning to what some of the right questions might have been, let me give two reasons why the above claim – even if true – carries little meaning.

First, self-citation is a far more serious problem than the authors are willing to acknowledge. It is also one that they admit to not having investigated. The data on Pakistani research papers shows that subtracting out self-citations drastically cuts down on actual citations – there are often 2-3 self-citations for every real one! The reader is urged to carefully study my email correspondence of last year with Dr. Atta-ur-Rahman (see appendix below) who, at the time when he was HEC chairman, had made similar claims that I disputed as being false. To interested readers, I have made available (in pdf form) the Thomson Scientific data that I have quoted in my correspondence here (1998-2003) and here (2003-2008).

Second, many authors of the relatively more cited mathematics papers from Pakistani institutions are not Pakistani nationals. High salaries offered to foreign faculty by the HEC brought to Pakistan a large number of well established mathematicians on short-term contracts from Russia, Poland, Ukraine, and China. This was probably a good thing to do – in spite of the difficulty they had in communicating in an alien language and their consequent inability to teach well. Their papers, however, do not reflect mathematics in Pakistan. One sees a similar phenomenon in Saudi Arabia where foreigners are principally responsible for the kingdom’s large number of papers and citations.

In my opinion, instead of focusing on marginal matters, serious research on the state of Pakistani higher education, and of changes therein, would have first established appropriate metrics, and then sought answers, to the following key questions:

1. What is the quality of teaching in Pakistan’s public universities? There is often only a weak correlation between formal qualifications and subject competence, so simply counting PhD degrees is not very helpful in answering this. Far too frequently one sees professors of English who cannot speak or write a single sentence of grammatically correct English, physics professors who are stymied by Newton’s Laws, and biology professors whose knowledge is frozen in some pre-Darwinian age. But does such basic incompetence exist at the 20, 50, or 70 percent-level? Higher? Lower? What evidence exists that the HEC’s reforms improved the situation?
2. Is there evidence that there has been improvement in the selection process for students in public universities, or that of the quality of their graduates? Proof of the latter, judged by asking employers or assessing performance in international tests, would be a clinching argument for the success of HEC reforms.
3. Do campuses enjoy greater academic freedom, more seminars and colloquia, less violence by extremist campus groups, a pleasanter and more relaxed ambiance, and greater transparency in faculty selection? Surely these are critical to any reasonable assessment.

To get answers to questions like these requires extensive field work, and I certainly do not fault the authors for not doing this. But I was surprised that the Nature article, as well as the lead author’s response, merely says that the HEC’s experiment had critics, without citing any specific articles or the substance of those criticisms. There is not even a passing reference to the failed nine-university multi-billion dollar mega-project, tons of unused scientific equipment purchased for unknown reasons, dubious attempts to fund “Quranic Science” (that had to be hastily abandoned after the scheme was exposed), and the explosion in academic corruption set off by per-paper payments. Surely, these should not be brushed aside as “collateral damage”. In another country, those who massively squandered public money would have been thoroughly investigated by independent commissions, not praised for small things.

The Bottom Line: Well-functioning universities are the products of a complex organic and evolutionary process that is internal to a society. Money and facilities matter, but it is much more important for a university to have a forward looking world-view, an open environment, high ethical standards, a sense of collegiality and shared sense of purpose, and good governance practices. Sadly, the Nature article did not even mention these as significant.

Join the Discussion!
What are the correct metrics to measure higher education reform in Pakistan?

Related Posts:

1. Nature’s Coverage of Higher Education Reform in Pakistan: Comments by Dr. Atta-ur-Rahman
2. Nature’s Coverage of Higher Education Reform in Pakistan: A Conversation with Athar Osama
3. Pakistan’s Higher Education Funding Holds Many Lessons for Developing Nations: Nature

Editors Note: Prof. Pervez Hoodbhoy is a well-known Pakistani nuclear physicist and political-defence analyst. He is the Professor of High Energy Physics, and the head of the Physics Department at Quaid-e-Azam University, Islamabad, Pakistan. Prof. Hoodbhoy is a vocal critic of HEC’s policies and their impact. The views expressed in this article are solely those of the author and do not necessarily reflect the views of STEP.

This correspondence between Dr. Atta-ur-Rahman and Dr. Pervez Hoodbhoy is reproduced below with consents from both parties. It is exclusively concerned with a public matter, has no private content, and is largely focused upon the importance of self-citations.

Dear Dr. Pervez
I have probed into the situation at QAU, and it is quite the opposite of what you claim, as evident from the total publications and the total number of citations, as per above attachments. The HEC was established in October 2002. The funds started coming through from July 2003. The subsequent impact of HEC programs on research publications and citations is indeed amazing. In 2004 the number of publications was only about 120—by 2007 it has risen to about 380—a 300% increase! The citations in 2004 were about 800—-by August 2008 they have increased to about 2200 although we still have 4 months to go before the year ends—I suspect that it will be about 3200 by the end of the year—a 400% increase!

I shall be separately sending you a report after removing self-citations. QAU is also now included in the most cited institutions in the world (please see attachment)—-this was not to 4 years ago.

Please be fair and objective in your assessments. We may have made some mistakes, but much good has happened.
Kind regards
Atta

Dear Dr. Pervez
Following my other email to you this morning, I am now enclosing the citations of QAU after removing self-citations.As you would see, these have risen from only 84 in 2004 to 1413 in 2008 (with still 4 months to go). These will probably be around 1900 by the end of the year—a spectacular ten-fold growth! Dr. Naim has kindly had these searched, so if you have any queries about them, you may like to interact with her. She tells me that the situation is similar in many other universities—a long period of stagnantion
during the 1990s followed by a burst of activity in the last 4-5 years.
Kind regards

Atta

Dear Dr. Atta,

I am sorry about the late response to your three emails. First, thank you for doing whatever you did – the HEC’s notification of 14 July 2008, which specifies 40 percentile as the GRE passing marks, finally reached QAU departments today (without comment from the administration). In these times one has to be grateful for small things…

Regarding your email and the numbers cited therein: I can understand that you would like to feel upbeat about QAU having improved itself as a result of massive infusion of HEC resources. I too would very much like good things to happen, but perhaps one should not allow wishes to become conclusions.

Looking at the data that you had sent me and your inferences, I decided to do a little independent investigation using exactly the same database (ISI Web of Science) and exactly the same keywords (see attachments to this email). Here are the findings:

Period: 1998-2003
Number of papers published by QAU authors in the above period: 631
Number of citations to date: 4540
Number of citations to date with self-citations removed: 2,817

Period: 2003-2008
Number of papers published by QAU authors in the above period: 1482
Number of citations to date: 3667
Number of citations to date with self-citations removed: 1258

Some obvious inferences:

1. There is absolutely no evidence of real citations having increased; if anything, the numbers up to now show the contrary. While the number of real citations of papers published 2003-2008 may increase somewhat with time, currently they are quite a bit less from the earlier period when the HEC and your incentive system did not exist. Please remember that citations are cumulative over years. I have tried to use exact descriptions in the figures cited above. If I am wrong in any detail, or if I have missed something essential, I would like to be corrected. Unfortunately the data does not at all support your rather optimistic remark of “a spectacular ten-fold growth!”
2. The above data also indicates the disturbing fact that most of the time QAU authors cite themselves. Subtracting self-citations drastically cuts down on real citations – there are 2-3 self-citations for every real one!. Looking more minutely at the ISI pages, one also notes that many citations are by other members belonging to the same or other QAU departments. So the number of genuine citations gets cut down even beyond the numbers quoted above (2817, 1258)!

Just to get scales right, here are a few citation numbers from the Stanford Spires data base for Pakistani physicists over the length of their careers:
a) Riazuddin: 1479
b) Ahmed Ali (DESY, Germany): 9873
c) Abdus Salam: 14103

3. You are correct that the number of publications over equal (5-year) time periods has more than doubled relative to pre-HEC times. But this is clearly in response to the monetary incentives offered by PCST/QAU. A publication fever now grips our universities. It is difficult to defend the case that the number of papers published is proportional to the amount of research done.

You are, of course, free to have the data I have quoted above rechecked and I would be happy to answer any question that arises. Finally, please note that publications and citations were not central to my earlier expression of dismay at the quality of QAU education.

What bothers me much more is the lack of analytical and problem-solving abilities of our Ph.D graduates, some honourable exceptions aside. Poor performance in the GREs is one indication of the rot. This fact has indeed worried you a little, as you indicated in an earlier email to me, but I do wish you could understand the real gravity of the situation.

Regards,

Pervez

Dr. Atta-ur-Rahman’s comments on this post can be found here.

This is not the first time that Pakistan has been named in these ratings recently. In fact, Pakistan’s record has been very consistent since March 2008, the earliest ratings that are available on the website. Here’s a listing of Pakistan’s mention in the ‘rising star’ ratings:

• March 2008: Engineering, Mathematics
• May 2008: Materials Science
• July 2008: Engineering
• September 2008: Computer Science, Engineering, Materials Science, Mathematics, Plant and Animal Sciences (5 areas!)
• November 2008: Engineering
• January 2009: Computer Science
• March 2009: Computer Science
• May 2009: Materials Science, Plants and Animal Sciences
• July 2009: None

Quaid-e-Azam University in Islamabad, the country’s top university in terms of the number of publications per year, has also been recognized as a ‘rising star’ institution, in Jan 2009 and July 2008 issues, both times in the area of ‘Engineering’.

The ratings are based on the largest percentage increase and not the absolute numbers, and therefore, cannot be used to quantify research productivity in absolute terms. However, they definitely demonstrate the trend of a substantial increase in international publications from Pakistan compared to previous years. It is very healthy that a number of different areas are covered in these past two years, showing an across the board enhancement of research productivity.

While there has been a lot of debate on the effectiveness of HEC’s reforms in higher education, at least one thing is clear: the increased emphasis on research, largely due to HEC’s programs, has started to bear fruit. These are hard numbers here, based on data by the company that maintains the largest scientific citation index in the world, and cannot be easily refuted by the nay-sayers.

Acknowledgement: Thanks to Dr Usman Qazi for alerting me to these ratings.

STEP: You have worked as a professor, as a dean, as an industrial entrepreneur and now as a policy-maker in the government. Which of these roles did you find the most rewarding?

SN: I find the one that I’m doing now the most rewarding because of its ability to influence so many factors pertaining to education in Pakistan. But I do miss the university environment, especially the interaction with students. I’ve always loved teaching and being in the classroom. In fact, I sometimes catch myself talking to my colleagues as if I’m lecturing them! So that’s definitely something that I do miss. There is a freedom in being a professor that is simply not available in any other job. When I’ve had it with administration, I can always go back to being a professor.

STEP: Coming to your work in the HEC, very few people managed to survive the change in government. How did you manage to survive the cut?

SN: That is a good question. I have felt that the true challenge to an organization is to move beyond the individual realm, which Pakistan seems to be suffering from, and to build an institution. The true test would be when we would survive an actual change in administration and see whether the HEC is an institution or just a bunch of individuals. So we all took that challenge very seriously. We had been working since the very beginning to institutionalize everything and we had done everything purely on merit. So we just had to buckle down and weather the storm and allow our work to speak for itself. And that is what has happened with the grace of God. We continued to work in an absolutely merit based manner and were not partial to any entity whatsoever and ultimately our work was recognized and we were supported by the new administration as well. Begum Shahnaz Wazir Ali has played a critical role in this transition and without her leadership we would not have survived the cut.

STEP: After the change in government, the HEC’s budget was drastically cut. I would like to congratulate you on having these budget cuts reversed recently. How did you manage to bring this about?

SN: It was actually a very long struggle. We had in fact been working with the World Bank for many years trying to convince them to support the government of Pakistan for the higher education sector. The World Bank had never given any budgetary support loan to any county in the world to support higher education; it had always been lower education or technical education. So we had to work long and hard with them to convince them of the successes of the HEC program and show that this is viable. That came through and helped the government to go over the budgetary shortfalls.

STEP: Do these loans from the World Bank come with any terms?

SN: The terms are financial in nature and do not have any policy implications. This is a budgetary support role, thus the World Bank is supporting the higher education program of the Government of Pakistan and is not funding any one particular initiative. The financial terms are soft and have a ten year grace period, thirty year repayment period and the interest rate is half a percent.

STEP: During your tenure at the HEC, what have been the HEC’s three biggest achievements?

SN: I think that probably the biggest achievement is bringing respect back to the university, through the university and the university faculty; the university as an institution being recognized by the country and its people as something to be proud of, to be nurtured and to be built. That, I think, is something to be proud of. The second would be the rebirth of research in our universities. Whereas universities were indulging in research in a sporadic, individual based manner, now they have taken on the research agenda with much more vigor, with an across the board response to it and this is something that the entire country is involved with. The third thing would be the introduction of a system of education that is compatible with the best in the world, which involved changing the bachelors and masters degree structure, restructuring the four year undergraduate program, introducing a course-based masters, and introducing a course-based PhD, so that the academic structure in Pakistan is compatible with the best structure that is practiced in the world. No Pakistani graduating now can feel that they have gone through a system that is inferior to anybody. The system is not inferior, the system is compatible. Now what matters is the work that you do.

STEP: Have these achievements had any tangible effects up till now?

SN: Most definitely there have been tangible results. We can talk about simple numbers, in terms of the number of students who are engaged in higher education in the country. This was estimated to be 2.6% of the youth population when the HEC started operation and it is now crossing 4.7%. Then, there are a number of disciplinary opportunities that are available to students. Five years ago when you completed an FSc degree in engineering and you did not get into one of the few engineering campuses, basically you had to sit back and figure out, “which college am I going to go to? What am I going to do?” Now it is a completely different scenario. If you’re coming from the IT stream, there is IT, computer science, and telecommunication, but you can also receive undergraduate degrees in physics, mathematics, etc. We have also been able to get people to voluntarily come back to Pakistan as academics. There were many Pakistanis doing PhDs abroad by themselves. They are now choosing to follow an academic discipline in Pakistan. We have a tenure track system, respectable salaries, and a good environment to work in the universities. People are coming back to our public universities and are taking teaching positions over here. In terms of research publications, Pakistan had on the order of 700 or so international publications per year throughout the 90s. In 2008, we crossed the 4000 mark in terms of publications. Just Quaid-e-Azam University alone, which is the number one research university in Pakistan, is crossing 500 publications in one year.

Then, in terms of quality assurance, [we have set up] the entire structure, the mechanism of quality enhancement cells, and accreditation councils. I mean, we never had any mechanisms for checking the quality of computer science programs in Pakistan. We had two year bachelor degrees, three year BCS, four year degrees, BITs, etc. I mean, you think about it, and a nomenclature in the structure existed. Today, we have a single four year undergraduate program. We have an accreditation council and we have a rating system, which is going ahead and checking programs. So in terms of quality, we are a far cry from where we were.

STEP: The HEC’s achievements that you listed are confined entirely to a very small percentage of the population. How do you think the HEC has positively affected society at large?

SN: I think with the society at large, our greatest impact has been on the parents, who are looking to provide university education to their children. We are not having a large impact on the policy-makers, which is where much more work needs to be done. That would mean much greater emphasis on the social sciences and the humanities, and continued capacity building of the faculty members, so that they take a leadership position with the development of policies. You see, universities should have an inherent leadership position in society. They are moving towards that and in some cases they have. I mean, when I switch on the TV and I see, for example, a sociology professor from NED [University] talking, it makes me happy that the media would come to faculty members and look at them as experts. But our society is very large and at a policy level you have got to understand that only one percent of the youth (17 to 23 years of age) or a little more are enrolled in universities. Ninety nine percent are not. Three to Four percent are in colleges, distant education (Allama Iqbal Open University), and other programs such as internal programs, etc. That would still leave 95% out of the loop. That is what is not being understood and that is the biggest battle Pakistan faces, the 95%; what are you going to do about that?

The work of HEC has begun to bring dignity back to our Institutions of higher learning. Society is looking up to them for leadership, industry is coming to them for talent and a solution to their problems. Some new breakthroughs are beginning to surface so the future looks bright.

STEP: But the 95% should also benefit from the HEC work. Shouldn’t research focus on local problems?

SN: Research should focus on local problems. That is something that needs to be done; Research needs to be locally relevant and it needs to be of an international standard. This is where I think that we need to do more work in incentivizing our faculty members to take up local challenges. For example, a manufacturing technology center was put up in UET Lahore, looking at the small to medium scale manufacturing industry that is concentrated in Lahore. An automobile center in Karachi, a date palm research center in Shah Abdul Latif in Khairpur, earthquake engineering machine center in Peshawar … are some examples of how HEC is supporting locally relevant research. But the faculty needs to be motivated to do that, and that requires more work.

STEP: Perhaps the HEC should restrict the research grants to research that focuses on locally relevant problems.

SN: You see there is a supply and demand issue. Pakistan’s problem, and this is something that one needs to really, really understand, is that of capacity. We just don’t have that many researchers, who are doing research. So of these people who are applying for research now, if you want to put in an additional constraint of forcing them to focus on local problems only, then you will have to define what is locally relevant and what is not. That appears to be an easy problem to solve but in practice it would be just about impossible. There are only degrees of relevance here. All research is relevant to Pakistan but the time frame in which it may impact local conditions is going to be different. But then this does not mean that you have to move away from this challenge of getting research to be locally relevant. Let’s say you need to work on the Thar coal fields. You need a large number of experts in various inter-disciplinary fields to actually focus on such problems. You have to understand that practical problems are extremely, extremely complex at times to address. Solving them at times requires you to have a very large team of experts which may not exist at any one university. What you can do and where universities can make a contribution, is to focus on development a little more; that means much more in terms of applied research. Let’s say there is a small factory producing some goods. It is possible to undertake a project to automate the factory units. Maybe you design software to speed things up, you look at the business processes – this is much more in the applied domain, which sort of gets out of the university domain. So this is a challenge in which each of us in the universities and academia has to ask ourselves that question of “how are we going to be relevant?”

STEP: How does the HEC plan to face this challenge?

SN: One thing we are doing in the social science domain is the introduction of thematic research, where themes are identified by groups of local experts, and this is something that we are also going to be introducing in the scientific fields. Our first goal was to get research going, to get people in that mindset – thinking and being inquisitive and innovative. Now, there is the question of starting to channel these resources so that there are, for example, technologists, who can look at food, agriculture, and ways of harvesting, [which is] one of the big areas of possible economic benefit in Pakistan. We could also start looking at issues of health for Pakistan and this has already started to happen. The Center of Excellence in Molecular Biology, for example, is looking at hepatitis and what we can do to locally manufacture interferons to treat this disease. Similarly, things are also beginning to happen in the direction of producing genetically modified crops for Pakistan. It is now necessary to take the next step, identify a number of themes and support research in those identified areas.

STEP: When do you think the common man will start seeing the benefits of this research?

SN: The common man has started to benefit. For example, this interferon developed by the Center of Excellence in Molecular Biology is ready for human testing. The problem is that the protocols for human testing in Pakistan are not yet well defined, since this is the first time this has happened. As soon as that happens, we are talking about millions of hepatitis infected patients being able to be treated by a medicine developed in Pakistan. Similarly, there are other products, and these days we are trying to put together an intellectual property portfolio for Pakistan, where we would categorize these [products] and try to get local or foreign investments going. The next challenge is to get research out of universities and into the industry domain. There are other interesting possibilities, such as salt-tolerant crops, which can be used as fodder for animals and can grow in millions of acres in Baluchistan. There are some vaccines that are being developed for animals, poultry, cows, etc., with huge benefits. So there is this kind of work which is beginning to show up, which is of commercial interest and will impact the common man. I am hoping in the next five year time-frame that these products, at least some of them, should be in the market.

STEP: A major challenge the HEC faces is the “elitist” quality of Pakistan’s higher education. How do you plan to face this challenge?

SN: Well, the way we are doing it is multi-pronged. We are taking higher education out of the main city framework so you have universities now in Malakand, in Hazara, in Sargodha, in Gujrat…so we have enlarged that domain. And we provided them with the latest IT technology so that they are linked. The other thing is this entire issue of a need based scholarship framework, in which we are providing scholarships and developing the capacity of need assessment so that you can actually identify who requires financial aid and who doesn’t require financial aid. The third path is the outreach path, where you are actively going into the rural schools or the suburban schools of Multan or Dera Ghazi Khan and building the capacity of students from the schools in these regions to take entrance examinations of top universities in Pakistan. All three approaches together are addressing the key issue of equitable access to higher education in Pakistan.

STEP: What were some of the policies or plans that did not do as well as you were hoping?

SN: I think the issue of governance has been the most difficult because, you may not know it, in the early days there was a great talk about the Model University Ordinance. There was talk about restructuring the existing universities and bringing in new governance structures. It was not accepted by the faculty and that was one of the areas where we did not make much headway. That is the only one I can think of at this time.

STEP: What about the HEC’s plan to build universities with foreign aid?

SN: Yes, that was another policy put in place much later, to build these mega universities. It is not on the table anymore. We have decided to scrap those projects and to rethink them and redesign these as well. They were too big and too ambitious, and if you combine that with the worldwide economic recession, then it is clear that the project is no longer viable.

STEP: One of the justifications for these mega-universities was that because of the PhD scheme we will have many PhDs returning to Pakistan without institutions to absorb them. Now that the mega university project has been scrapped, do you think we have the universities to absorb the incoming PhDs?

SN: The current fraction of PhD qualified faculty in our universities is hardly crossing 24% at this time. Three out of four faculty members do not have PhDs even currently. The student-teacher ratios are high and the demands are extreme. We have new campuses opening up, we have a rapidly expanding university system growing at a rate of about 15% per year. Now to cater to this growth of 15% alone would require an addition of about a thousand to twelve hundred teachers per year. So there is no shortage of capacity to absorb PhD qualified faculty in our universities. Also, as research is taking a hold in our universities, research groups in different areas are beginning to form in different institutions. This in turn feeds the demand for additional highly qualified faculty which is going to be available in the future.

STEP: So they will have jobs when they return?

SN: Yes they have jobs; in fact the HEC guarantees them a job. Any new PhD will be hired by the HEC for the first year if they cannot find a job; so that is not an issue.

In part two, we talk with Dr. Naqvi about the mandate of the HEC, the future of the organization, and how policies are crafted at the HEC.

Editor’s Note: Minor edits have been made to the article since it was first published.

Related Post: A Conversation with HEC Executive Director Dr. Sohail Naqvi: Part 2

IEEE INMIC is held every year and INMIC 2009 will be the 13th in the series. INMIC has become Pakistan’s flagship technical conference with a broad scope, thereby inviting interest of a large audience. The conference targets research presentations by academic and professional researchers, and also includes a series of tutorials, enabling participants to learn about the latest trends in technology. Research contributions are expected from the participants, covering various disciplines under IEEE’s domain, including technical papers, panel discussions, tutorials and project exhibitions. For all submitted papers, the review criteria include significance of the problem, novelty, clarity, completeness, and accuracy.

As a venue for research, a multi-topic conference like this one is necessarily going to have substandard work: researchers with good work aren’t going to publish here because the audience will not be able to fully appreciate their contribution, and the related scientific sub-community (who would cite their work) aren’t going to be in attendance and therefore won’t be aware of the research. A Google scholar search of citations for this conference shows an abysmal citation record: not a single paper in the 13 year history of the conference has managed more than 10 citations. While this Google scholar search may have missed some papers, it is a reasonable indicator of the influence of a conference.

I suppose an argument could be made that there is value in its function as a gathering place for researchers in Pakistan, but if this is really the purpose of the conference, a symposium with invited speakers may be more worthwhile than a parade of poor research. In any case, the scientific community in Pakistan may have now reached a critical mass making these multi-topic conferences somewhat obsolete. Thoughts?

Here are a few photos from Umair Asim’s blog, a Lahore-based ameteur astronomer, who had volunteered his time (and his telescope!) for these events.

The audience at District Public School, Okara, at the Teesra Falakayati Mela

Another view of the audience at Okara

Umar Asim explaining the craters of the moon to the audience

A banner annoucing the event

The live viewing is preceeded by a general-interest lecture on astronomy. Second from left in the front row is Dr Saadat Anwer, president of the Khwarizmi Science Society

An earlier event at Phool Nagar, the Doosra Falakayati Mela

Setting up the telescope at Government High School, Phool Nagar

Here’s a video of the first event at the Punjab University, Lahore…

UPDATE: Dr Sabieh Anwer, the general secretary of KSS, just informed me that the next Falakayati Mela will be at the historic Rohtas Fort near Jehlum, which has the oldest school of the area from pre-independence period. For more pictures of the earlier events, have a look at this page. An article on the Phool Nagar event was published in The News.