The self-assessment of FURG's graduate programs is linked to institutional self-assessment and external assessment processes. At the institutional level, the Self-Assessment Committee (CPA), with the support of the Institutional Assessment Directorate, coordinates the self-assessment processes linked to the National Higher Education Assessment System (Sinaes) (Law No. 10,861, of April 14, 2004). In this context, students in master's and doctoral courses carry out an annual assessment of faculty members, considering the didactic aspects of the graduate teaching activities, and faculty members evaluate the classes, infrastructure, and management. The data produced are evaluated by the Self-Assessment Committee (CPA) and the Internal Assessment and Planning Committee (CIAP) of the Academic Units where the graduate courses are located, supporting the planning processes and the construction of the annual action plan of the Units, which are linked to the objectives and strategies of the Institutional Development Plan of FURG. These bodies are also coordinated with the Stricto Sensu Graduate Committee, formed by all the coordinators of the stricto sensu graduate courses and by the Graduate Directorate of the Pro-Rectory of Research and Graduate Studies. This Committee evaluates the demands of the programs, the results of the self-assessment and external assessment procedures, and the proposals for the creation of new courses, in addition to constituting a forum for the exchange of experiences, discussion, critical evaluation, and mutual assistance among the course coordinators. The Office of the Vice-President for Research and Graduate Studies also has an Advisory Committee, made up of experienced graduate faculty members, to evaluate and review Sucupira reports and to assess proposals for new courses. Furthermore, DIPOSG holds annual meetings with the Graduate Programs, focusing on programs with grades 3 and 4, for which the meetings involve the entire faculty.
In addition to these procedures, IN PROPESP No. 1/2024 specifically deals with the monitoring of strategic planning and self-assessment procedures of PPGs. Since 2023, the coordinators have been completing a report with information regarding the annual self-assessment of the PPG via sistemas.furg.br, which allows the Directorate of Graduate Studies (DIPOSG/PROPESP) and the Directorate of Institutional Assessment (DAI/PROPLAD) to monitor it. Furthermore, it allows PPGs to maintain a history of the internal assessment activities of their courses, which may support the preparation of the CAPES four-year report. Furthermore, to assist in the evaluation and planning procedures of each PPG, the Pro-Rectory of Research and Graduate Studies, in agreement with all PPGs, contracted the Stela Experta-PG Platform via PROAP resources, which allows coordinators to monitor the quantitative variables of their PPG, in addition to providing a comparative analysis with other PPGs, enabling the organization of guided Strategic Planning.
The PPGComp program for the Master's and Doctorate in Computer Engineering is unique in that it approaches Computing from a different perspective, unlike traditional Computer Science courses. We are interested in the scientific challenges associated with the creation of computational artifacts to solve complex problems. On the other hand, the course does not end with computer design, but rather has its core in the concept of algorithms and their efficient implementation. These two characteristics make PPGComp a unique program where new theories, methodologies, and applications must be proposed to solve complex problems of nature and engineering that today can only be addressed by computational devices.
The faculty is composed of a mix of professors with extensive experience in training human resources at the postgraduate level, productivity fellows, research group leaders, and young researchers with great potential, energy, and availability to face new challenges.
In this context, based on the evaluation of the last four years, and aiming to obtain a higher level of concept than the current one, the program's self-evaluation procedures were consolidated. These procedures are systematized, monitored, discussed, and improved in monthly PPGComp meetings. In addition, all evaluation metrics are calculated automatically using a set of software systems, some developed by the program itself.
1. Research Workshop of the Center for Computational Sciences. Held at the beginning of each four years, the event aimed to present an overview of the research of the academic unit since its creation and to analyze the integration of researchers over time, to motivate the engagement of newly hired professors, and to foster research activities. A tool developed by the Information Management Research Group was used to visualize the academic collaboration networks of the members of the groups registered in the CNPq directory and the relationships with the permanent members and collaborators of PPGComp. In addition, the evolution of scientific publications produced each year was demonstrated, and trend curves were analyzed. This workshop is one of the diagnostic tools for the strategic planning of the program presented in Section 1.3.
2. Annual self-assessment workshops. Organized in the second half of each year of the four years, the objective was to discuss the results and planning of actions related to scientific production and fundraising for the execution of research projects. Each professor of the program presented a list of submitted articles, containing the title, authors, publication vehicle, bibliometric indexes in international databases, Qualis stratum, and status (published, accepted for publication, under review, or rejected). A list of submitted projects was also presented, containing information on the title, team, notice or call, amount of funding, funding agency, and status (contracted, approved, under review, rejected). The results achieved are discussed by the members of the PPGComp and guide the planning of activities, considering the following two years, focusing on the number of students supervised, increase in scientific production, and fundraising, mainly financial. The submission of projects for the CNPq productivity grant call is also encouraged and planned by all members of the program, which resulted in the current composition, where one-third of the PPGComp professors are productivity grant holders, and the rest have merit, although they were not considered due to budgetary restrictions.
3. Assessment of postdoctoral leave. Since its creation, the Center for Computational Science has encouraged its staff to take time off for academic qualifications. During the four years, four PPGComp faculty members completed postdoctoral internships at the following institutions:
- Albert-Ludwigs-Universität Freiburg / Germany
- Universidad Pública de Navarra / Spain
- Austrian Institute of Technology - AIT, Austria;
- Universidade Federal do Rio Grande do Sul - UFRGS / Brazil.
These experiences enriched the research carried out in the areas of Robotics and Artificial Intelligence and effectively enhanced the internationalization of the program. Today, FURG has already established international agreements that have allowed student and researcher mobility between these institutions and the signing of joint projects. These collaboration networks have also benefited the supervision and co-supervision of international graduate students. The scientific production reports clearly show the significant increase in interaction with coauthors from these institutions in recent years.
4. Mid-term seminar. In early 2023, after two annual self-assessment workshops, PPGComp held a mid-term seminar to analyze the quantitative and qualitative indicators achieved up to the middle of the four years, mainly based on the analysis of the Computing Area Document and the discussions held by the Forum of Coordinators of Graduate Programs in Computing and the Brazilian Computer Society. Among the decisions taken at the seminar, the most notable were the greater rigidity in the program's permanence rules, the planning of the second two-year evaluation period, the longer-term strategic planning, and the process of re-accreditation of faculty members.
5. Re-accreditation of faculty members. The number of permanent faculty members and collaborators of the PPGcomp varied throughout the four years, as shown below:
2021 - 20 permanent faculty members + 1 collaborator = 26
2022 - 16 permanent faculty members + 5 collaborators + 1 visitor = 22
2023 - 16 permanent faculty members + 5 collaborators + 1 visitor = 22
2024 - 16 permanent faculty members + 5 collaborators = 21
Following the results of the 2017-2020 four-year assessment and the dismissal of two permanent professors, given their approval in a public selection process at another HEI, it was decided to carry out a new, even more rigorous re-accreditation process, based mainly on scientific production indicators, prioritizing the quality of intellectual production. In early 2021, professors were ranked according to their scientific production weighted by the number of co-authors, and only 16 professors were retained. Other evaluation criteria were also considered in the same period, such as: workload in PPGComp disciplines, number of projects coordinated with funding from companies, number of projects coordinated with funding from development agencies, number of dissertations defended, number of ongoing supervisions, productivity grants, among others. The vast majority of these indicators were highly correlated with weighted scientific production. In 2022, other calls for proposals were launched that resulted in the accreditation of 5 collaborators and the hiring of a visiting professor. It was also decided that there would be no new notices in the four-year period in order to keep the teaching staff more stable and analyze the performance of strategic planning.
It is worth noting that although the permanent faculty has been slightly changed, the 16 permanent faculty members who have been working since 2022 are a subset of the 20 professors from 2021.
This process contributed significantly to improving the quality of the program's research and scientific output.
COMPARATIVE EVALUATION WITH OTHER PPGs WITH DOCTORATE
The analysis presented below considers public data from 2021 to 2023 available on the Sucupira Platform in March 2025, comparing FURG's PPGComp with 40 other Graduate Programs of the academic modality evaluated in Computer Science, all with doctoral courses. There are 8 PPGs rated 7, 4 PPGs rated 6, 12 PPGs rated 5, and another 16 PPGs rated 4. PPGComp indicators are presented below, along with the median and position in the ranking when compared to the other programs. These complete results and graphical analyses can be consulted at https://sistemas.prpg.usp.br/apoema-pg/view/fa82b28847befee985cec1dbd8cbef6c. It is worth mentioning that this procedure was carried out annually, systematically, throughout the four years. The results were monitored in the Self-Assessment Workshops. The data presented below can be viewed graphically in Annex 08: Comparative analysis - Sucupira (2021-2023).pdf.
Student training indicators:
- Dissertations completed in the three years with scientific publication by students (Master's with publication) = 25; median = 28; position = 26
- Dissertations completed in the three-year period (Masters graduates) = 44; median = 48; position = 22
- Average number of dissertations per permanent advisor (Masters per permanent faculty member) = 0.85; median = 0.64; position = 9
- Average duration of the Master's degree (Time until defense) = 2.56 years; median = 2.85 years; position = 6
- Students enrolled in the Masters degree who have already completed = 42%, median = 33%, position = 8
The training indicators show that PPGComp produces more masters than half of the PPGs analyzed. Furthermore, it does not compromise the training time. Even though it only offers a master's degree compared to the other 40 programs, PPGComp is at the center of the distribution of master's with publications.
Scientific production indicators:
- Full papers in conferences = 234; median = 192; position = 13
- Full papers in conferences by permanent advisor = 4.5; median = 2.57; position = 3
- Distribution (Gini coefficient) of conferences by advisor = 0.43; median = 0.46; position = 12
- Annual average of journals weighted by Qualis and by permanent advisor (general index - Qualis strata A1 to B4, score relative to A1) = 1.61; median = 1.31; position = 12
- Distribution (Gini coefficient) of journals by advisor (restricted index - Qualis strata A1 to A4) = 0.46; median = 0.48; position = 16
Due to the fact that conferences do not have unique identifiers like journals, an automated analysis was performed comparing the conference titles extracted from the publications of all PPGs in the same evaluation period (2021-2023) and the titles extracted from the conference spreadsheet with Qualis in CA-CC available at https://ppgcc.github.io/discentesPPGCC/pt-BR/Qualis/. A software for approximate matching was developed, using textual similarity based on trigrams. The implementation made use of the pg_trgm.similarity() function available in the PostgreSQL DBMS. The similarity threshold was experimentally adjusted in order to minimize the error between the automated evaluation and the manual evaluation performed with the PPGComp data itself. The following indicators position PPGComp concerning the same 40 PPGs from the previous analysis. Bar graphs were generated and attached to this proposal to facilitate the visualization of the results, especially concerning the other PPGs.
- Annual average of papers in conferences weighted by Qualis and by permanent supervisor (general index - Qualis strata A1 to B4, score relative to A1) = 2.09; median = 1.38, position = 7
Analyzing journal articles in conjunction with conference papers:
- Annual average of journals + conferences weighted by Qualis and by permanent supervisor (weighted iGeral - Qualis strata A1 to B4, score relative to A1) = 3.7; median = 2.91; position = 7
It is worth noting that, for this comparative analysis, the indicators were calculated with all permanent faculty members, including PPJ and PPS. The reason for keeping these faculty members in the calculation is that it is not possible to know how many junior or senior researchers will be declared by the other 40 PPGs analyzed in addition to PPGComp.
The values presented show that PPGComp is among the best of the 41 programs analyzed, being much better than several PPGs rated 5, 6, and 7 in the various comparison criteria. In many of the criteria, it appears in the top positions of the ranking. It is one of the programs that publishes the most in conferences, but maintains a good rate of publications in qualified journals. It is worth mentioning that this comparison did not include the data from 2024 because these will only be publicly available after the Coleta is processed, after the submission of this report.
Regarding the distribution of productivity among faculty, the data show that PPGComp is at the top of the 2nd quartile, considering both journals and conferences. In other words, it is one of the programs that least depends on a small group of faculty members to make up the productivity indexes.
RESULTS OF QUANTITATIVE INDICATORS OF EVOLUTION IN THE FOUR YEARS
The following items present the results for each evaluation criterion, considering quantitative indicators. The data were calculated using the production registered in Sucupira in December 2024 (2021-2023 triennium) plus data from the Lattes CVs of PPGComp professors. It is worth mentioning that this same analysis was carried out in the mid-term seminar, and the monitored results were used to readjust the strategic planning, proposing actions for the second evaluation biennium.
- Permanent professors with post-doctorate degrees. We increased by 50%, going from 6 in 2020 to 9 in 2024. Today, there are already 11.
- CNPq productivity scholarship holders. We maintained the quantity but increased the quality, raising the level of two researchers, going from one PQ 1D level, one DT 1D level, two PQ 2 levels and one DT 2 level in 2020 to one PQ 1C level, one DT 1C level, two DT 2 levels and one PQ E level.
- Articles in journals (Qualis strata A1 to B4). The calculation of this and other indicators considered the Qualis estimate based on the current Scopus or WoS percentile.
[2017-2020] Annual average = 37.5
[2021-2023] Annual average = 42.3
Growth of 13% when compared to the previous evaluation period.
- Annual average of journals weighted by Qualis and by permanent advisor (weighted iGeral - Qualis strata A1 to B4, score relative to A1). Growth of 58% to the previous four-year period.
[2017-2020] Annual average of 1.02
[2021-2023] Annual average = 1.61
- Full papers in conferences (Qualis strata A1 to B4). Reduction in quantity by 27% when compared to the previous four-year period.
[2017-2020] Annual average = 107
[2021-2023] Annual average = 78
This reduction is justified because publishing efforts have focused on international conferences and high-impact journals, thus increasing quality over time, as demonstrated in the indicators with Qualis estimates.
- Annual average of conferences weighted by Qualis and by permanent supervisor (weighted iGeral, Qualis strata A1 to B4, score relative to stratum A1). Growth of 7% compared to the previous four-year period.
[2017-2020] Annual average = 1.96
[2021-2023] Annual average = 2.09
- Annual average of publications (journals + conferences) weighted by Qualis and by permanent advisor (weighted iGeral, Qualis strata A1 to B4, score relative to stratum A1). Growth of 26% to the previous four-year period.
[2017-2020] Annual average = 2.98
[2021-2023] Annual average = 3.75
- Number of active scholarship quotas.
2022 = 4 Capes DS + 1 CNPq + 12 projects = 17
2023 = 4 Capes DS + 3 CNPq + 20 projects = 27
2024 = 9 Capes DS + 2 CNPq + 15 projects = 28
Public funders of project grant quotas:
ANP
CAPES
CNPq
EMBRAPII
FAPESC
FURG
Private funders of project grant quotas:
Auros Robotics
Climeggi
Data For Company
DDOS
DENSO
Freedom
GoAwake
GreenNext
I. Melito LTDA
Insaight MedTech
Longevité
Medlife
Natubras
Petrobrás
R&F Tecnologia
Raiaviva
Sarmento
SEBRAE
STIHL
Tedesco
TuboMaq
UBOTS
- Number of ongoing research projects. Growth of 37% over the four years.
2021 - 66
2022 - 76
2023 - 80
2024 - 90
These projects represent a total of R$31,804,237.96 (thirty-one million, eight hundred and four thousand, two hundred and thirty-seven reais, and ninety-six centavos) invested in research and development activities by PPGComp professors in the four years.
- International agreements and collaborations: detailed in Section 3.3 Internationalization, insertion (local, regional, national) and visibility of the program.
DISCUSSION
The quality of research produced for a stricto sensu postgraduate course has as its main indicator the quantity and quality of intellectual production. These were the main criteria criticized in the evaluation in the 2014-2016 triennium and, precisely for this reason, they were the ones that received the most attention from the program since 2017. A great effort was made to increase qualified productions, detailed in the quantitative indicators presented above. During the last four-year evaluation, PPGComp demonstrated that its production rates were better than the average of courses with a score of 4. In these four years, the program further boosted the growth rate, presenting consistent indicators over the years. Currently, PPGComp is in the first and second quartiles of the distribution of most production indicators when compared to other PPGs with doctoral courses, ahead of many programs with scores of 5, 6, and 7.
The awards received by PPGComp faculty and students, detailed in Sections 2.2 and 3.1 of this report and the evaluation form appendix, also demonstrate the quality of the research carried out.
The number of R&D&I projects associated with different funding agencies and private companies has been growing rapidly. The accreditation of the iTec/FURG Unit - Center for Robotics and Data Science - https://www.itecfurg.org - with the Brazilian Company for Industrial Research and Innovation - EMBRAPII has allowed it to capture abundant financial resources for research, development, and innovation, which have practically doubled every year. All Itec/FURG projects rely on PPGComp faculty and students, which tends to increase significantly with the recently approved Doctorate course in Computer Engineering.
The international collaborations carried out within the scope of the research lines prove the PPGComp's decision to provide our students and faculty with opportunities to experience international excellence groups. In addition, the PPGcomp registered several exchange and internationalization actions in the four-year period, detailed in Section 3.3 Internationalization, insertion (local, regional, national) and visibility of the program.
The efficiency of student training is evidenced by the indicators number of master's degrees and average training time, which were excellent in the same comparative analysis. Regarding student training, the increase in interaction with undergraduate students is also noteworthy. This interaction is expressed by the large number of undergraduate students involved in research activities and the good distribution of publications with students, in addition to a significant and regular number of awards for these students.
Research involving technology transfer projects and the creation of startups at the OCEANTEC Technology Park, highlighted in Section 3.2 Economic, social, and cultural impact of the program, was also considered as a result of the self-assessment.
ASPECTS TO BE CONSIDERED
PPGComp's productivity results have continued to grow over the current four-year period. When compared to all PPGs with doctorates, regardless of grade, that is, including programs with grades 5, 6, and 7, PPGComp ranks in the top 2nd quartile for journals and in the 1st quartile for conferences. In addition to having a performance equivalent to consolidated national and international reference programs, the balance in the distribution of production among faculty members is better than in 3/4 of the programs with doctorates, considering both journals and conferences.
Despite all these excellent results, the coordination team identifies that scientific dissemination and the internationalization of the program should be improved. There are still a few visiting professors and foreign students. Goals, actions, and internationalization indicators were detailed in the strategic planning.
Comparative analysis - SCIndexBR(2025)