WEB OF SCIENCE İLE ATIF TABANLI ARAŞTIRMA DEĞERLENDİRMESİ Eğitmen: Metin TUNÇ This presentation is an introduction to InCites - the new web-based research evaluation tool from Thomson Reuters. Thanks to Philip Purnell of Thomson Reuters for providing the content for this session. 1

Niçin Araştırma Performansını Değerlendirelim? Güncel ekonomik iklim Artan seçici olma ihtiyacı Örnek veri grupları / metodolojiler: Verilen hibelerin değeri ve sayısı Ödüllerin sayısı (ör: Nobel Ödülleri) Denklik değerlendirmeleri Yayın sayısı Atıf sayısı Kriterlere güre normalleştirilmiş bibliyometrik incelemeler Ülkelerin önemli bilim kuruluşları bibliyometrik değerlendirmeleri önemserler. Today, governments, administrators, and others with a stake in research are having to distinguish those projects and researchers that deserve prioritization over the rest. More and more funds are limited yet published research continues to grow. As such, administrators can no longer rely solely on personal knowledge and experience to identify the best research. New methods are needed to help professionals in these areas evaluate research and now government, research institutes and libraries in many countries are using bibliometrics to inform and support their decisions. Thomson Reuters has been using bibliometrics for 30 years now and almost all these groups are using Thomson Reuters publication and citation counts and the benchmark metrics we derive from these to evaluate the relative performance of individual researchers, research groups or departments, universities and research institutions, journals and even entire countries.

Uygulamada İhtiyaç; Kurumların karşılaştırmalı bilimsel performans ölçümü Ulusal ve uluslararası ortalamalar ile karşılaştırma ve gelişim hedeflerinin belirlenmesi Bilimsel teşvik sistemi için kriter oluşturma Bilimsel faaliyetlere yönelik kısa ve uzun dönem stratejik planlama Kurumun ulusal ve uluslararası bilimsel iletişim/paylaşım incelemesi, teşvik kriterleri ve stratejik planlaması. Bilimsel teşvik sistemleri ile gelişmesi olası yapay kendine atıf incelemeleri ve gerekli kriterlerin oluşturulması ... .. Today, governments, administrators, and others with a stake in research are having to distinguish those projects and researchers that deserve prioritization over the rest. More and more funds are limited yet published research continues to grow. As such, administrators can no longer rely solely on personal knowledge and experience to identify the best research. New methods are needed to help professionals in these areas evaluate research and now government, research institutes and libraries in many countries are using bibliometrics to inform and support their decisions. Thomson Reuters has been using bibliometrics for 30 years now and almost all these groups are using Thomson Reuters publication and citation counts and the benchmark metrics we derive from these to evaluate the relative performance of individual researchers, research groups or departments, universities and research institutions, journals and even entire countries.

Araştırma Değerlendirmesi; Veri Kaynağı Thomson Reuters çözümleri Web of Science’dan sağlanan tutarlı ve güvenilir verileri değerlendirir. Tutarlı, otorite ve anlaşılır biçimde tanımlanmış veriler anlamlı ve güvenilir istatistikler için temeldir. Bu nedenle kaynak olarak Web of Science esas alınmıştır; Dünya çapında 90’dan fazla ülke ve 3200’den fazla kuruluş tarafından kullanılan altın standartlı atıf kaynağıdır. Fen bilimleri, sosyal bilimler ve beşeri bilimler alanlarında 11.000’den fazla dergi ve binlerce konferans bildirisi içerir. 109 yıldan fazla arşivi ile benzersiz atıf verisi içerir. Tarafsız, süreklilik testinden geçirilmiş ve sıkı bilimsel dergi seçim süreci Etki değerinin (Impact Factor) kaynağı – En yaygın olarak kabul edilmiş dergi atıf göstergesi InCites exclusively uses the Web of Science data. A general rule of statistics is that the result of any statistical analysis will always be limited by the quality of the data used to conduct the evaluation. Therefore, it is crucial to use a collection of papers and citations that all have a comparable value and that is the reason for using the Web of Science as the source of data.

THOMSON REUTERS Yayın Seçim Politikası Yayınlanma standartları Peer review, Editorial conventions Editoryal içerik Bilimsel alanlara özel mevcut bilgi birikimine katkı Çeşitlilik Uluslararası ya da bölgesel etkisi olan yazarlar, editörler, danışmanlar... Atıf analizi Yazarlar ve editörlerin ön çalışmaları Each jounal in the Web of Science has been individually selected based on consistently applied strict criteria, such as an adherence to Good Publishing Standards and application of a peer review process. Editorial content is evaluated by subject experts to ensure value to the field, and the international diversity of the authors and editors is considered as well. Finally, a citation analysis is conducted to illuminate the influence and impact of the authors’ and editors’ other research publications.

Web of Science ile Kurumsal Performans İncelemesi CANLI DEMO InCites exclusively uses the Web of Science data. A general rule of statistics is that the result of any statistical analysis will always be limited by the quality of the data used to conduct the evaluation. Therefore, it is crucial to use a collection of papers and citations that all have a comparable value and that is the reason for using the Web of Science as the source of data.

INCITESTM Kurumsal Performans İncelemesinde Uzman Çözüm InCites exclusively uses the Web of Science data. A general rule of statistics is that the result of any statistical analysis will always be limited by the quality of the data used to conduct the evaluation. Therefore, it is crucial to use a collection of papers and citations that all have a comparable value and that is the reason for using the Web of Science as the source of data.

Anlaşılır Biçimde Tanımlanmış Verisetleri Son 30 yıllık araştırma çıktılarımızın durumu nedir? The first thing to decide when considering InCites is to define the dataset you want to analyze. Typically an institution would select the papers published by their own researchers over a specific time period. Alternatively, papers published worldwide on a specific subject area can be prepared. In Incites, the publications selected for analysis are found in The Source Articles Listing. In this example we can see the metrics that are calculated for each paper. These include a times cited count, a second generation citation count and the mean second generation citation count for the citing articles. InCites also provides the mean, or expected citation count for a paper of the same document type, in the same database year and in the same journal and we call this the Journal Expected Citations (JXC) so looking at the second paper on the list, we see that publications classed as ‘Article’ in 1992 in the journal ‘Nature Cell Biology’ on average have received 92.64 citations. We can also see the expected, or mean number of citations for a publication in the same subject category and the same year which in this example was 33.66. This paper has received 528 citations so it is a highly cited paper compared with both the subject category and the journal.

INCITES Kurum Yayınlarına Atıf Yapan Makalelerin İncelemesi The citing articles are those papers that contain the citations to the source papers and these are also included in your dataset for analysis. The citing articles are presented in a bar chart showing the year in which they were published and then in a table ordered by number of times cited. The table can be reordered by the number of references made to the source articles, the publication year, article type, author, journal or article title in alphabetical order. Here we can see the number of times these citing papers were in turn cited, and these are termed second generation citations. It is interesting to see the number of second generation citations the citing papers have received because it provides an indication of the quality of the citing papers and thus the long term, or indirect influence of your source papers.

INCITES Kuruma Ait Yayınlar ve Bunlara Atıf Veren Yayınların İncelemesi Kurum Makalelerinin Performans İncelemesi So far we have seen the list of source articles and the list of citing articles. Each of these groups of papers can now be ranked through a variety of parameters. First we will see how to rank the authors of the source papers in your dataset.** Atıf yapan makalelerin incelemesi

Bizim En Etkili Yazarlarımız Kimler??? The Author Ranking of source articles shows a list of all the authors that have appeared on the source articles in this dataset. By default the authors are initially ranked in order of the number of citations their papers have received and this can be an indicator of the overall influence the authors have had. The authors can also be ranked by total articles published, average citations received per article published, h-Index, actual citations received versus expected citation rate for journal and for subject category, mean percentile or in alphabetical order.

En Aktif Yazarlarımız Kimler Re-ranking the authors by the total number of papers published gives us a measure of research productivity for each author. It is interesting to note the change in the order of the authors, for example ‘Hawari, J’ has the fourth highest publication count but ranks only 17th in order of citations received.

Görsel Öğeler ve Normalleştirilmiş Verilerle Yazarlarınızı Karşılaştırın The reports are all interactive so we can compare the performance of two researchers side-by-side, which cannot be done with the Web of Science. The authors B Massie and JHT Luong have similar H-index scores and the citation counts are within the same range but when we look at the performance of their papers relative to the fields in which they publish we see that B Massies papers have been on average cited 2.27 times the mean citation rate in their field whereas JHT Luong’s papers have been cited 1.31 times the average rate. Therefore, although JHT Luong has published more papers than B Massie, it is B Massie’s papers that have performed better compared with the global benchmark for the field and year of each paper.

İŞBİRLİĞİ: KURUMSAL İNCELEME The Web of Science captures and indexes all authors on all papers and all addresses of those authors. Ranking the institutions of your source papers gives a picture of the institutions with whom your faculty collaborates.

Kurumumuz hangi kurumlarla işbirliği gerçekleştiriyor? The institution ranking lists the institutions given in the addresses of all the authors on the source papers and shows the extent of co-authorships, or collaborations undertaken by your faculty. This is a very good way to see at a glance the extent of national and international collaboration between institutions and authors. Again, these are initially ordered by number of citations those collaborating papers received which indicates the total influence of these papers.

Bu işbirliklerinin hangileri en değerlileri? However, we don’t only need to know with whom our faculty collaborates but how did those collaborations perform? In order to see the relative performance for a multidisciplinary set of papers, it is necessary to normalize the citation counts for the subject category in which they were published and for the year in which they were published as older papers have had longer to accumulate citations than more recent ones. In this example **, collaborations with the Max Planck Institute for Molecular Physiology received 71 citations ranking them in 89th position but those papers were cited on average in the 13th percentile and therefore had more impact per paper relative to their field and antiquity than other collaborations with a higher publication and citation count.

Bizim Uluslararası Etkimiz Ne? Ranking the citing articles provides information on those researchers who are influenced by your faculty. These are the countries listed in the addresses of authors on the citing articles and shows the geographical influence of the original dataset and is of significant interest to research organizations. So in this case, 879 papers from authors in Switzerland have cited the source papers and those papers themselves received an average of over 45 citations each so the Swiss citing papers can be considered highly influential papers. The time series of these citations can help to quantify the effect of policies or events as they took place. One might imagine a hypothetical visit of a member of faculty visiting Switzerland in the year 1996 which resulted in a quantifiable increase in citations from Swiss researchers to your organization. Burada Neler Oldu?

POLİTİKA DEĞİŞİMİNİN ETKİLERİNİ İNCELEYİN Politikaların gelişimi çerçevesinde bilimsel performans doğrudan etkilenir. Thomson Reuters InCites

Daha Detaylı Analizler İçin Daha Detaylı İstatistikler So far we have looked at different ways of analyzing source articles and citing articles from the entire dataset but in many cases users want to analyze collections of papers from individual researchers, groups or departments and to compare them globally or with one another. InCites makes this possible through the ‘Limit Report Results’ tab which is really an advanced refining tool. *2* Users can choose to analyze different subsets of source or citing papers and create different report types and compare rankings.

Detay Sorulara Detaylı Yanıtlar Bizim kimya araştırmacılarımız hangi dergilerde yayın yapmışlar? Desteklediğimiz araştırmacılar neler yapmış, ne yayınlamış? Mühendislik alanında en etkili makalelerimiz hangileri? Hangi işbirliği grubumuz matematik alanında en etkili? Mühendislik alanında hangi ülkeler/kurumlar bizi en çok etkilemiş? Limiting reports by various parameters can provide answers to questions like: IN WHICH JOURNALS DO OUR BIOLOGISTS PUBLISH? WHERE DO BIOLOGISTS WHO ARE INFLUENCED BY US PUBLISH? WHICH ARE OUR MOST INFLUENTIAL ARTICLES ON ZOOLOGY? WHICH OF OUR COLLABORATIONS ARE MOST INFLUENTIAL IN MATHEMATICS? IN WHICH COUNTRIES DO WE HAVE MOST INFLUENCE WITH ENGINEERS?

Zaman Serileri ve Trend Analizleri Her bir yıl için araştırma verilerini sağlar. Neden 1993 yılı çok iyi? Bu politika işliyor mu? …ve 1996 zayıf? Incites’ time series trend graphs provide rolling snapshots of research performance for each year. This allows for a quick visual analysis that may reveal correlations with staffing, funding or policy changes within the organization. Personel ya da bütçe değişiklikleri atıf-yayın sayısı ile ilişkili olabilir.

INCITES Özet Ölçümler Atıf Ölçümleri Disipliner Ölçümler İşbirliği Ölçümleri The Summary Metrics report provides an overall snapshot for any set of articles you select and there is a summary metrics report automatically created for your source papers. These metrics are organized into three categories: citation metrics, disciplinarity metrics, and collaboration metrics. The citation metrics show basic metrics like the total number of articles in your chosen dataset, the number of citations they have received, the mean number of citations received by each article, as well as the h-index, median cites and number of second generation cites to this dataset. The disciplinarity metrics are measures of the concentration or dispersion of a group of papers over a set of categories. Incites uses the journal category classifications of the dataset for these calculations. You can use the disciplinarity and interdisciplinarity indices from the overall Summary Metrics report as baseline measures to evaluate the degree of specialization for specific entities Collaboration metrics indicate the extent of collaboration among authors, organizations, and countries.

INCITES Özet Ölçümler Ortalama değerlere göre karşılaştırmalar Additional summary metrics include the percentage of cited vs uncited articles, the mean percentile and actual vs expected citations for the journal and category. The “% Articles Cited/Uncited” pie chart depicts the proportion of cited articles and uncited articles in the set and can be used to identify the proportion of papers in the dataset with little or no influence. We ought to remember that very new papers always have a lower chance of having received citations so when using this metric it is important to compare paper collections from the same time period. The Mean Percentile gauge indicates the mean of the percentiles for all of the articles in the dataset. The gauge indicates where the mean percentile is in relation to the expected mean percentile, which is always 50. So any mean percentile lower than 50 means this collection of papers has performed better than average. Category Actual/Expected Cites is a ratio that provides a normalized measure of category-level performance for a multidisciplinary set of papers. Journal Actual/Expected Cites is a ratio that provides a normalized measure of journal-level performance for a set of papers from multiple journals and categories. If the number of total citations is the same as the total journal expected or categoery expected cites, then the ratio equals 1.

Raporları Oluşturma ve Aktarma Reports that have been limited can be exported, so clicking on ‘Keep this version’ presents the option to send the report as an email attachment or in the body of the email. Or alternatively we can save the ranking as a report in our workfiles.*2*

Raporları Oluşturma ve Aktarma The reports can be visualized and saved manually as pdf or as Microsoft Excel files

INCITES Klasör Oluşturma – My Folders “My Folders” allows you to save and manage your reports and folders according to your needs, including setting various read/write permissions for sharing with colleagues in your institution.

Özet Analizler, dünyanın önde gelen araştırma değerlendirme çözümünden sağlanan tutarlı ve otorite verilere dayanır. Bölümler ya da yazarların sorularına cevap alabilecekleri ve kendi alt veri setlerini oluşturabilecekleri, uygun hale getirilmiş veriler sunar. Karşılaştırmalı yüzdeler, ölçümler ve bilgiler ile kendi performansınızı karşılaştırabileceğiniz anlamlı veriler içerikler sunar. Standardize ve normalleştirilmiş ölçümler ile etkili, şeffaf, adil ve sistematik bir rapor çıkarmanıza olanak sağlar. Web tabanlı erişim sayesinde kurumdaki herkesin kullanması olanaklıdır. In summary InCites provides: Authoritative, consistent data. A tailored data set and the ability to create your own sub-sets and associated metrics provides for specificity, answers to questions at a local level. Context around the data, such as baselines and percentiles, gives the metrics genuine meaning, comparative value. Standardized and normalized metrics and formatted reporting tools for efficient, transparent, fair and systematic research evaluation. Great flexibility in scheduling and running current and future reports supports organizational efficiency. A Web-Based tool, accessible by any number of select users within your institution. 27

Teşekkürler! Detaylı bilgi için; Sorunuz varsa: http://scientific.thomsonreuters.com/training/ Sorunuz varsa: isi@mikrobilgi.com.tr Thanks for viewing this presentation. To see more recorded training modules visit our training site. If you have additional questions or would like more information contact the training team using the second URL listed here. Don’t forget that you can download these slides by clicking on the attachments tab in the brainshark viewer.