You are here
Citation Statistics Report by the IMU
Mon, 2008-06-16 04:20 -Sanjay Govindjee
Here is a report on Citation Statistics written by the International Mathematical Union written in cooperation with the ICIAM and IMS.
Given all the discussion on impact factors and h-indicies, I thought many people may find this report interesting.
| Attachment | Size |
|---|---|
 PressRelease-en-2008.pdf |
90.32 KB |
| CitationStatistics-FINAL-1.pdf |
416.94 KB |
»
- Sanjay Govindjee's blog
- Log inorregisterto post comments
- 14188 reads
Comments
Citation Based Evaluation is very dangerous for economy!
This paper pointed out many very good issues about the citation based evaluation it missed many other deficiencies.
1. If we assume all authors had ideal review,every paper is based on a few key papersand number of other papers. in current citation based system all papers get same credit. and it is not true
2. due to last deficiency if you do some thing which is alittle useful for many papersis very better thandoing something great for a few papers.
3. there isno limitation for number of referencesand many low quality papers has too much references and change our citation data base significantly.
3. citation based system have aparadoxical nature. if we call higly cited papers high quality papers and lowly cited papers low quality. must of papers are low quality and since much of references comes from these papers,quality of high quality papers is deteremines from low quality papers.
4. there is no parameter to check thefinal of influence of a bunch of papers on industry and society. then a network of low quality papers can cite each other.
6.reviewers of many ISI journals dont check even content of the papers good, references are next step.
7. alot of number of references comes from introduction partand in this part many authors just like to justify their work.many of them use any paper which help them in this justification.
8.H factor is good for select nobel prize winnerswhich has a number of hi quality papers it is not good for ordinary researchers. their citation diagram is very different in act. (i checked!). (attention even Einestein was an ordinary researcher)
9. Some books and review papers summurize papers and then others refer to books and review papers. then many of papers will lost because they summurized in books or review papers.
10. when some author or inistitution becomes famous people follow his works and it gets more citation (halo effect)
This discussion is good, but we need to make clear facts
1) Highlycited scientists are really good, in most cases.
Check thewww.isihighlycitedscientists.comdatabase. You can argue about it, but it is sure that you will find the names of the 250 best in each of the 21 disciplines generally very good --- some are nobel prizes, political leaders and so on
2) it is sad that out of 5250 total, 4014 are from US, and just more than 1200 from the rest of the world!!! This is a real dominance.... and has effects in many ways. If you are not publishing from US, it is harder.
3) n.3 country comes UK with about 400, and Switzerland has an impressive 144 for its size...
4) Italy is only at 81, not very good at all. A group of them was formed in 2003, which has a web site,www.gruppo2003.it,and I am helping them creating a web online journal
5) if anyone in imechanica wants to create the group of Engineers highlycited, there are 43 in Europe, and 250 in USA. I can attach the list it is public, no secret.
6) something interesting that has NOT been done, is to correlate this list with the more recent H-index. If we do that, it has scientific interest. Clearly, I have noticed that in the top 81 italians, there are people with 81 or 66, in the range of Nobel prize...
7) Zhigang Suo is not yet included in any discipline. Maybe a disvantage of his work being spread among different disciplines. I remember him getting upset when I would tell him that in engineering alone is h factor is about 18, whereas the total is maybe 43. Another effect, Zhigang, and here is no story to argue about, is that you are not yet included here. We should all help Zhigang getting "challenged" to get it there, otherwise he would feel bored having achieved so much already!
8) I start with list of ISI highlycited Engineers in Europe (43)
Åström, Karl Johan Lund Institute of Technology/Lund University Sweden
Beven, Keith John Lancaster University England Ecology/Environment Engineering
Biersack, Jochen P. Hahn-Meitner-Institut, Berlin Germany Engineering
Billings, Stephen A. University of Sheffield England Engineering
Bimberg, Dieter Technische Universität Berlin Germany Engineering Physics
Charpak, Georges CERN Switzerland Engineering
Chen, Sheng University of Southampton, Highfield England Engineering
Clarke, David W. University of Oxford England Engineering
Crisfield, Michael A. Imperial College London England Engineering
Crochet, Marcel J. Universite Catholique de Louvain Belgium Engineering
Dagan, Gedeon Tel Aviv University Israel Engineering Ecology/Environment
Dubois, Didier Institut de Recherche en Informatique de Toulouse France Computer Science
Eckstein, Wolfgang Max-Planck-Institut für Plasmaphysik Germany Engineering
Fleck, Norman A. University of Cambridge England Engineering
格洛弗基斯剑桥大学英国注意事项neering
Groeseneken, Guido IMEC - Katholieke Universiteit Leuven Belgium Engineering
Hopfinger, Emil J. LEGI France Engineering
Hornik, Kurt Wirtschaftsuniversität Wien Austria Engineering
Isidori, Alberto Università degli Studî di Roma La Sapienza Italy Engineering
Jones, Kevin C. Lancaster University England Ecology/Environment Engineering
Kemmer, Josef KETEK GmbH Germany Engineering
Lutz, Gerhard Max-Planck-Institut fnr Physik Germany Engineering
Mallat, Stéphane G. École Polytechnique France Engineering Computer Science
Morari, Manfred ETH Zürich Switzerland Engineering
Payne, David N. University of Southampton England Engineering
Philipps, V. EURATOM Association Germany Engineering
Prade, Henri CNRS France Computer Science Engineering
Rodi, Wolfgang Alfons Universität Karlsruhe Germany Engineering
Roth, Joachim Max-Planck-Institut für Plasmaphysik Germany Engineering
Sauli, Fabio CERN Switzerland Engineering
Schwarzenbach, René P. Vorsteher Institut für Gewässerschutz undWassertechnologie Switzerland Engineering Ecology/Environment
Sharir, Micha Tel Aviv University Israel Engineering Computer Science
Sigmund, Peter University of Southern Denmark Denmark Engineering
Stoica, Peter Uppsala University, Department of Information Technology Sweden Engineering
Tvergaard, Viggo Technical University of Denmark Denmark Engineering
Unser, Michael École Polytechnique Fédérale de Lausanne (EPFL) Switzerland Engineering
Vetterli, Martin École Polytechnique Fédérale de Lausanne (EPFL) Switzerland Engineering
Weilhammer, Peter CERN Switzerland Engineering
Winter, Jörg Ruhr-Universität Bochum Germany Engineering
Wold, Svante Umeå University Sweden Engineering
Wright, Richard F. Norwegian Institute for Water Research Norway Ecology/Environment Engineering
Wyatt, Richard England Engineering
Zienkiewicz, Olgierd Cecil University of Wales Swansea Wales, UK Engineering
Regards,
michele ciavarella
www.micheleciavarella.it
A discussion with one of highly cited researchers.
I have one discussion with one of these highly cited researchers. and i asked him about his highly cited paper and he answered me very trustly. i just used a method developed by others to supply needs of others who did not know that. it is just like a movie, when we see a movie we just see the stars but we dont see other people who supply this stars. must of research achivements are evolutionary but when itchange the PHASEwe acknowledge people we see in the phase change. not the people who had role in that evolution. It is even true about nobel prize winners.
Many of Peaks are still peaks , i am not sure about other peaks!
I hope mathematicians find a way to model scientific society with a more exact network model and will prove deficiencies of current system more exactly. but in very simple mathematics just assume the main results is first funtion and is multiplied by a random function caused by these errors. final functions is what we see know as citation numbers. if peaks be strong enough they remain peaks at final functions but if many of peaks coincide we can not be sure that two function exactly corrolate! in other words i know the most citeds are good are they best?!. are'nt others better than them? this is the question.
well even peer review can make you a star in science in many way
Roozbeh
to become a start in science fast, the traditional way is that you are a student of a start scientist in science. That speeds up publication. Imagine you co-author with a Nobel prize winner. Who is going to say your paper is bad?
There are similarities with the movie Star industry as well. With ups and downs. And scandal. You remember the Korean guy who claimed he cloned first a sheep?
See for example this interesting story:
http://www.wired.com/science/discoveries/news/2002/09/55410
The Undoing of a Star Scientist
Wired News Report
09.26.02
(Editor's note: This story originally was credited as being written by
Michelle Delio. In fact, the Associated Press contributed a significant
portion of the reporting. Ms. Delio added her own, independently
researched material and composed the following report, which now does
not carry her byline).
Jan Hendrik Schön seemed to be a modern day alchemist.
He appeared to be able to manipulate atoms with amazing grace. He
seemed to be able to force electricity to do seemingly impossible
tricks, and claimed he'd found ways to make working molecule-sized
electronic components.
The
results of many of his experiments contradicted known laws of physics.
Other scientists were unable to duplicate Schön's work. Some said the
young man must have magic hands.
But not all.
In May,
Lucent Technologies, which runs Bell Labs where Schön worked, responded
to growing queries about Schön by appointing an independent committee
to investigate. The results were released Wednesday, validating what
许多物理学家已经提出几个月go.
The committee believes Schön made up or altered experimental data at least 16 times between 1998 and 2001.
朗讯庵野unced the firing of Schön on Wednesday. But some questions still remain.
Schön, 32, published 80 reports in prestigious journals over the last
two years, more than most scientists produce in a lifetime. Some
scientists think the established scientific peer-review process failed
by allowing some of those papers to be published.
But many others believe that Schön's story proves the merits of standard peer-review procedures.
"Yes, misconduct occurred," said Malcolm Beasley, an applied physics
professor who chaired the committee that investigated Schön. "But the
normal processes of science ferreted it out. The system worked."
But the investigating committee did find that there is a need for
"clear, widely accepted standards" of professional responsibility to
ensure the veracity of published scientific results.
Bell
Labs isn't the only scientific institution to be smacked with charges
of fraud recently. The Lawrence Berkeley National Laboratoryannouncedin June that it had disciplined a researcher for fabricating the results of a published experiment.
"It's a very good thing for people and organizations in science to look
at this," Beasley said. "It's not something that the community has
thought through."
Bell Labs spokesman Saswato Das said
laboratory officials will be taking steps to prevent misconduct,
including encouraging extended review of new research before it is
published and clarifying supervisors' responsibilities for reviewing
all research papers.
The signs of Schön's fraud were writ large, but perhaps are only readable in hindsight.
Schön substituted data in his published papers, supplying fake graphs
that he told investigators "looked better" than the real graphs.
He also used the same graph in a dozen papers on different experiments.
And his data were often far too precise, far beyond reasonable
statistical probability.
Schön acknowledged that the data are
incorrect in many of his papers but said that some problems occurred
due to "honest mistakes."
But investigators noted that "the
recurrent nature of such mistakes suggests a deeper problem. At a
至少,Schön showed reckless disregard for the sanctity of data in
the value system of science."
Some who worked with Schön at
Bell Labs said he is a quiet, almost shy man -- "totally unpretentious,
totally unlike the egotist you'd imagine would be capable of such
acts," said one research assistant, who spoke on condition of
anonymity.
"People think that he must have cracked under the
pressure ... so much was expected of him," the assistant said. "It's so
sad. All his work is now suspect."
"It's two years that have
to be erased," Princeton University physicist Lydia Sohn said. "A lot
of us had hoped that this was new stuff and that it was going to be
really cool for the future of electronics."
Sohn was one of the scientists who alerted Bell Labs to problems with Schön's work.
Schön did not respond to telephone messages left at his New Jersey home Wednesday night.
In a statement responding to the committee's findings, Schön said, "I
tried to communicate the science that described the experimental
findings and that I was convinced of. Although I have made mistakes, I
never wanted to mislead anybody or to misuse anybody's trust.... I
truly believe that the reported effects are real, exciting and
reproducible."
The Associated Press contributed to this report.
michele ciavarella
www.micheleciavarella.it
Even Nobels get turned down
Mike you write, "Imagine you co-author with a Nobel prize winner. Who is going to say your paper is bad?"
I would say lots of people.
While it certainly can help to have famous co-authors, i can say from experience on grant review panels and in the paper reviewing process that even Nobels (or other top people) have their work brutally critisized and even rejected. Sometimes being famous seems to hurt, as people often have a bit of Schadenfreude in taking down famous people in an anoymous manner. At best, I think having big names in an author list gives you a bit of deference, in that if the reviewer does not understand something in the paper/proposal they will spend extra time trying to understand it before writing a negative comment (that may reveal themselves as less bright than the authors).
作为一个奇闻,我将添加汉斯Einst的故事ein (Albert's son). Hans was a Civil Engineering professor at Berkeley and at one time wrote a paper dealing with the Navier-Stokes equations. In the paper, he apparently made a fundamental error. This paper then became one of the most highly cited papers in the field because many people took joy in being able to point out that an Einstein made an error. This story is also a bit of caution in assuming that all highly cited papers are good. (I have never verfied the accuracy of this story but I do trust the person I heard it from.)
Prof. Dr. Sanjay Govindjee
University of California, Berkeley
Both of these two experiences are result from one point!
7. most of references comes from introduction part and in this part many authors just like to justify their work. tell us an story, many of them use any paper which help them in this justification. i think it is better to just calculate those whichUSEDin the paper in citation calculation.