1 Introduction

A report is a concise and organized document in which reasoning and activities related to a subject are recorded, with the fundamental objective of communicating them, either to the author themselves at future moments, or to third parties. Reports, technical or technological reports, academic-scientific or academic-pedagogical reports are, among other types of documents, part of the category known as "grey literature," characterized as informal manuscripts with limited circulation and not subject to peer review (or in a broader sense, not edited or published through conventional public access channels). They are understood as elements capable of effectively contributing to the dissemination of knowledge and, as such, require clear criteria to enable their evaluation and an acceptable quality of the final text, as a form of selection (^{Botelho; Oliveira, 2015}).

Concerned with issues related to the proper communication and evaluation of this type of literature within Graduate Programs, a agency of the Brazilian Ministry of Education, “Coordenação de Aperfeiçoamento de Pessoal de Nível Superior” (Capes), in 2019, established the Working Group on Technical Production (WGTP), with representatives from all Capes Evaluation Areas, aiming to standardize criteria for the evaluation of what they referred to as the "Conclusive Technical Report - CTR," defined as follows:

A concisely prepared text containing information about the project/activity carried out, from its planning to its conclusions. It indicates in its content the relevance of the results and conclusions in terms of social and/or economic impact and the application of the knowledge produced (^{Brasil, 2019, p. 52}; translation ours).

In this regard, several meetings were held for deliberations and decisions regarding the WGTP Report (^{Brasil, 2019}), which included the following stages:

•Stage 1 – Evaluation of the results achieved in the study previously conducted by GT 06 CAPES (2015/2016); • Stage 2 – Consultation with all evaluation areas regarding the importance of each of the 62 technical productions listed in Table 1, conducted through an electronic form (Annex 1); •Stage 3 – Analysis of the relevance attributed by the areas to each technical production (Annex 2), and grouping of similar productions; •Stage 4 – Application of concepts to the Study and conversion of the most relevant technical production into Products and not the processes involved; •Stage 5 – Preparation of definitions, descriptive fields, and examples for each product, using the stratification criteria established in a previous study as a basis; •Stage 6 – Evaluation of the results achieved and preparation of the final report (^{Brasil, 2019, p. 10}; translation ours).

In light of this context and considering the importance of enhancing the production of grey literature within Graduate Programs, the present study aimed to analyze the WGTP Report in relation to the existing literature, in order to assess its suitability as a guiding instrument in establishing criteria and indicators that support the evaluation of technical reports as products suitable for qualified dissemination of technical and technological knowledge, with a scientific approach.

2 Literature review

According to ^{Soubhia, Ruffino, and Dessunti (2005)}, a report is a document in which systematic and organic descriptions of reasoning and activities about a topic are presented, requiring complex mental operations such as organizing, synthesizing, arguing, analyzing, contrasting, justifying, composing, and proposing. ^{Martens, Pedron, and Oliveira (2021)} propose the term "technological article" or "technical report" as

[...] works with a practical problem-solving approach, while maintaining scientific rigor [...] technological articles are professional productions aimed at presenting solutions to problems [...] They should also be written in language accessible to their audience, which is generally composed not only of academics but also of market professionals [...] (^{Martens; Pedron; Oliveira, 2021, p. 143}; translation ours).

^{Biancolino et al (2012)} define a technical report as the final descriptive product of applied research or technical production, prepared with scientific and methodological rigor. They further elaborate, particularly in the organizational field:

The technical report follows the format of a short scientific article (7 to 10 pages, including references). Therefore, the technical report should: (1) present the basic differentiation of describing an intervention instead of analyzing a conventional theoretical/empirical object; and (2) propose practical and concrete improvements/results that can be adopted in the future[...]with the support of theoretical references from the field (^{Biancolino et al, 2012, p. 299}; translation ours).

Moreover, as outlined by ^{Martens, Pedron, and Oliveira (2021)}, we understand that a manuscript containing known solutions to known problems, without apparent innovation, does not qualify as an academic-scientific technical report, but rather as an academic-pedagogical technical report, similar to the proposal of ^{Soubhia, Ruffino, and Dessunti (2005)}. In this context, ^{Biancolino et al (2012)} and ^{Martens, Pedron, and Oliveira (2021)} present the necessary characteristics of a technical report, characteristics that align it with a scientific article and the propositions presented by ^{Savickas (2009)}, ^{Motta (2017)}, and ^{Souza (2022)}, as depicted in Table 1.

As ^{Savickas (2009)} and ^{Motta (2017)} make clear, an article is a more rigorous type of report, originating from academic research and, we can add, subjected to peer review.

The scientific quality of a Technological Article does not differ from traditional articles[...] the difference between scientific and technological articles lies in the approach to research and, in some cases, the audience (^{Motta, 2017, p. 03}; translation ours).

As a rule, a report consists of pre-textual elements (cover, title page, and verso of the title page), textual elements (introduction, development, and conclusion), and post-textual elements (references, appendix, and glossary), varying in form and content of these elements in different sections of the manuscript, according to their purposes, intent, content, target audience, and communication medium (^{Soubhia; Ruffino; Dessunti, 2005}; ^{Savickas, 2009}; ^{Martens; Pedron; Oliveira, 2021}). Whatever the type of report, it has a fundamental purpose: communication, and a basic requirement: presentation of useful and pertinent information, written in a clear, concise, organized, and substantiated manner. As a result, the report aggregates information from various consultable sources, for the knowledge of third parties, and enables, through cognitive effort in synthesizing, the enrichment of the personal underlying concepts of the author.

The underlying concept, according to some authors(1-3), is the concept that serves as an anchor for new information that is incorporated into the individual's cognitive structure over the course of their history through processes of progressive differentiation and integrative reconciliation. As new information is assimilated, the underlying concept becomes stronger and more comprehensive (^{Soubhia; Ruffino; Dessunti, 2005, p. 270}; translation ours).

In this direction, ^{Soubhia, Ruffino, and Dessunti (2005)}, in a study analyzing 83 academic reports from students in two different groups of the Nursing course, used a methodology involving two readings (the first aimed to familiarize themselves with the content of the work, and the second involved highlighting and annotating the margins of the pages to identify the ideas of each topic). They organized raw data into a frequency table indicating the proportion of correct responses and the degree of significance and compared the performance between the two groups using the MannWhitney Test (x²) or Fisher's test. Relevant to our discussion, in summary, the authors propose, as part of the results, the need for a report to possess the characteristics presented in Table 2.

The postulations of ^{Soubhia, Ruffino, and Dessunti (2005)} find resonance in ^{Pereira (2012)}, ^{Lima (2013)}, and ^{Porto and Gurgel (2018)} when discussing elements related to the writing of an article. They are also echoed in ^{Martens, Pedron, and Oliveira (2021)}, who present characteristics necessary for a technical report, closely aligned with expectations for a scientific article. According to ^{Lima (2013, p. 51}; translation ours), "A good article should be written with clarity, precision, and fluency, essential conditions for readers to feel interested and capable of understanding its content." ^{Pereira (2012, p. 26}; translation ours) warns that "Scientific writing constitutes a complex presentation of facts and arguments, guided by an elaborate process of reasoning."

Echoing the attributes proposed by ^{Savickas (2009)} in relation to the scientific report, ^{Motta (2017)} and ^{Martens, Pedron, and Oliveira (2021)} further emphasize the necessary scientific foundation and the concern for clear and objective writing that is suitable for their audience, which largely consists of non-academic professionals.

The key criteria for evaluating a Technological Article are clarity and objectivity. Articles that get straight to the point certainly have a greater chance of advancing through desk review. Thus, it is important to make it clear in the title and abstract what the manuscript is about[...] (^{Motta, 2017, p. 03}; translation ours).

Additionally, according to ^{Savickas (2009)}, a good research report must possess clear and concise language that is accessible to non-experts, portray the state of the art of the addressed topic, and demonstrate an appropriate level of research maturity to constitute a significant contribution to the field. In summary, Savickas' propositions can be observed in Table 3 (above), which contains the necessary criteria for constructing a scientific report. Furthermore, ^{Savickas (2009)} highlights some characteristics that are commonly seen as mistakes, compromising the quality of the report. Despite these characteristics potentially being transformed into positive criteria, alongside others, we will follow the author's inclination and emphasize them as aspects to be avoided.

In this regard, ^{Savickas (2009)} points out errors such as: excessive empiricism devoid of robust theoretical foundation; inversion of syntactic and semantic relationships², inclusion of content from one section into another; textual prolixity with irrelevant, repeated, or extraneous content; baseless and unfounded speculations; reductionist argumentation omitting relevant content; use of primary expressions and clichés³, and underestimation of the reader's interpretive capacity, explicitly stating what can be easily inferred from the text⁴, terminological confusion; lack of acceptance or refutation of proposed hypotheses; insufficient information for assessment and replication of the work; excessive detailing of standardized and published methods; excessive heterogeneity in the studied sample; lack of clarity regarding instrument scores; inadequate or insufficient description of statistical analyses; unclear figures and tables with insufficient information for interpretation; absence of promised information in the method section; conclusions based solely on figures and tables without adequate textual description; and redundancy of information between text, figures, and tables. ^{Porto and Gurgel (2018, p. 115}; translation ours) also offer a warning about this last error: "Tables summarize a set of observations and should be self-explanatory, without repeating information already contained in the text."

^{Motta (2017)} presents additional criteria for the evaluation of technical reports:

In the introduction, clearly state the problem situation and/or opportunity for improvement related to the analyzed context (organization/government/social actors involved)[...]Include a brief diagnostic text about the situation/problem and/or opportunity, demonstrating mastery of the subject under study and the theoretical-scientific foundations supporting this diagnosis[...]Provide a concise description of the procedures used to gather data and relevant information for analyzing the situation[...]Present as an article a text that analyzes the problem situation and discusses possible alternatives for its resolution, innovation, improvement, or extrapolation[...]Conclude the text by demonstrating the proposal's contribution to organizations and/or society (^{Motta, 2017, p. 03}; translation ours).

In line with this, ^{Lima (2013)} presents a reference on what not to do when writing an article and defines a technical report as a manuscript that "Reports on an investigation already carried out and specifies the steps taken, the results obtained, the analysis/interpretation of data, and the established conclusions" (^{Lima, 2013, p. 58}; translation ours), corroborating many elements presented by ^{Soubhia, Ruffino, and Dessunti (2005)}, ^{Savickas (2009)}, ^{Motta (2017)}, and ^{Martens, Pedron, and Oliveira (2021)}, indicating the similarities between a report and an article regarding various aspects. Thus, a technical report, not unlike a scientific article (^{Pereira, 2012}; ^{Lima, 2013}; ^{Porto; Gurgel, 2018}), should be capable of convincing the reader of its truth by describing and arguing with clarity and objectivity, either through logical concatenation of its ideas or by supporting these ideas with practical experience or appropriate literature that can provide substantiation for the assertions.

In this sense, borrowing from the contributions of ^{Latour (1987)} when discussing the production of scientific articles, the text of a report, regardless of its type, needs to incrementally and parsimoniously "pile up" information to create a positive impact on the reader. To achieve this, the text should be friendly in its dialogue, allowing the author to dissolve into its lines while the reader recognizes themselves within it, desiring to have written it. The digestibility and fluidity of the text are directly linked to the style and language used. The use of syntax, jargon, and technical terms suitable for the target audience should contribute to clarity while avoiding incomprehensibility for other audiences. Furthermore, the text should anticipate possible objections from the reader, constructing an argumentation that can challenge any counterarguments through a clear, coherent, and well-structured reasoning.

Special attention should be given to the title of a manuscript due to its seemingly simple yet significant role in captivating the reader. According to ^{ABNT NBR 6023 (2018)}, the title is a word, expression, or phrase that expresses the constituent content of a document, while the subtitle, presented immediately after the title, provides extra clarification or supplementation to the title, aligned with the document's content. ^{Souza (2022)}, in a study aiming to answer the question "How have authors presented the titles of scientific works published in the proceedings of the XXI National Meeting of Research and Graduate Studies in Information Science (ENANCIB)?" analyzed 342 titles of scientific articles, of which 67.3% were found to be appropriate according to the adopted theoretical framework. This framework enabled the stratification of analyzed works into nine classes: 1) nonspecific title; 2) affirmative title; 3) interrogative title; 4) title with formulas or symbols; 5) title with abbreviations; 6) sensationalist titles; 7) titles with excessive use of terms; 8) titles containing the methodology; and 9) themes in place of titles.

In general, we can observe that the expectations for titles in scientific articles align with the propositions for titles in academic-scientific and academic-pedagogical technical reports, as observed in ^{Soubhia, Ruffino, and Dessunti (2005)}, ^{Savickas (2009)}, ^{Biancolino et al (2012)}, ^{Martens, Pedron, and Oliveira (2021)}, and ^{Souza (2022)}. The consensus lies in the requirement for titles to be precise, complete, concise, objective, informative, and specific, consisting of ten to twenty words, without sensationalism, power words, methodological description, illusions, abbreviations, equations, symbols, questions, affirmations, exclamations, periods, commas, or quotation marks.

Regarding the characteristics of writing articles, ^{Souza (2022)} also emphasizes the need for standardization, clarity, objectivity, precision, accessibility, and simplicity, aligning with what can be expected for report writing according to ^{Motta (2017)}. Consequently, we can infer from ^{Souza (2022)} that the function of writing in a scientific article also coincides with the function of a report: useful, clear, and direct communication.

Additionally, it's worth noting that according to ^{Savickas (2009)}, the "Conclusion" section, or what some may better define as "Final Considerations," is subsumed within the "Development/Discussion" section, while for ^{Soubhia, Ruffino, and Dessunti (2005)}, it constitutes a separate section. On the other hand, ^{Soubhia, Ruffino, and Dessunti (2005)} present the "Development" section as a single block, whereas ^{Savickas (2009)} breaks it down into "Method," "Results," and "Discussion," delving into the details of each of these subsections. Both authors, with minor differences, align with the propositions of ^{Martens, Pedron, and Oliveira (2021)} for the structure and content of technical reports. However, more than differences, we observe similarities and complementarities among these authors regarding the expectations for a report, with a more detailed element in ^{Savickas (2009)}, albeit with limitations in his original text. Thus, based on the literature presented, we embark on an analysis of the WGTP Report, the reflections of which we shall discuss next.

3 Discussion of Results

As a result of the Working Group on Technical Production (WGTP) within Capes, the Technical Conclusive Report (TCR) was defined as a product derived from activities carried out within Postgraduate Programs and subject to evaluation, according to the guidelines provided in the report produced by the WGTP (^{Brasil, 2019}). The term TCR encompasses various types, subtypes, and purposes of reports, all of which can be considered under the umbrella term of gray literature (^{Botelho; Oliveira, 2015}). Excerpts from Table 4 illustrate this typology.

Although the types and subtypes of reports subject to evaluation were explicitly defined within the context of the WGTP Report, there is no detailed clarification of how these typologies interact as elements capable of guiding the assessment of this category of literature. The interconnection is left to the subjective deduction of the user of the WGTP Report when used as a guiding tool for the evaluation of technical products by Postgraduate Programs in Brazil. Such a characteristic contrasts with the propositions of ^{Soubhia, Ruffino, and Dessunti (2005)}, ^{Martens, Pedron, and Oliveira (2021)}, and ^{Souza (2022)}, where textual clarity, including proper organization, cohesion, and coherence, is an essential element in a report. Without such clarity in the evaluation instrument, doubts arise about its existence in the results derived from its use, casting doubts on its validity.

When we compare the consulted literature (^{Savickas, 2009}; ^{ABNT, 2018}; ^{Souza, 2022}) with the results obtained from the analysis of the WGTP Report, it is possible to observe a failure to meet the assumption of clear communication, starting with the title and subtitle of the latter, "Technical Production - Working Group", which presents two shortcomings: it is a non-specific title and has a theme in place of the title⁵. This lack of clarity extends to the body of the text. Upon analyzing its content, we notice that the enrichment of the underlying concepts (^{Soubhia; Ruffino; Dessunti, 2005}) may not have been effective, given the conceptual mistakes and confusion throughout the text. In this sense, the Report exhibited shortcomings in terms of section organization, content conciseness, information clarity, theoretical accuracy, argumentative coherence, technical formalism, linguistic-grammatical rigor, validity of conclusions, and usefulness of results, contrary to assumptions outlined in ^{Soubhia, Ruffino, and Dessunti (2005)}, ^{Savickas (2009)}, ^{Motta (2017)}, and ^{Martens, Pedron, and Oliveira (2021)}.

As an example of the lack of organization, technical formalism, and linguistic-grammatical rigor, we have the proposed criteria for the evaluation of TCR in the section "Product Details according to Table 2” (our emphasis). In this section, the WGTP lists, through a form, criteria such as: the purpose of the Report, the degree of innovation in its content, the individual or collective nature of the report, and the nature and stage of its connection (if any) with research projects and scientific production of the author(s). They also request the non-obligatory declaration of resources, links, and applicability related to the technological product that led to TCR (^{Brasil, 2019}). This section does not effectively align with the section "Stage 4 - Concepts Applied to the Study and Conversion of Technical Production into Products" (our emphasis), as it is inconsistent with the reference table for works, which became "Table 3" in the same section, since the 185th meeting of the Technical-Scientific Council of Higher Education, as mentioned in the document itself. Furthermore, the titles of each section, up to section 5, start with the expression "Stage," in reference to the methodological stages. However, this formal pattern is not followed in sections 6 and 7, without apparent reason. According to ^{Pereira (2012)}, ^{Lima (2013)}, and ^{Porto and Gurgel (2018)}, this indicates a potential flaw in the manuscript, compromising its objectives.

Additionally, the fields of the TCR evaluation form, found in the section "Product Details (according to Table 2)," do not have a clear correspondence with the important information listed in the section "Stage 5 - Definitions, descriptive fields, and Description for each Product":

•Definition; •Examples; •Mandatory descriptive fields*; •Optional descriptive fields*; •Correlate with the options and fields available in the Sucupira and Lattes platforms; •Does the production need to be in the repository? •Classify and justify the productions and subtypes as technical or technological; •Determine if the production is the result of work carried out by the graduate program or if it is the result of individual work by the teacher, which would be done independently of whether the teacher is part of a program or not (^{Brasil, 2019, p. 21}; translation ours).

As an example of the lack of theoretical precision and argumentative coherence, we can mention the attempt to differentiate technical from technological products. In order to delimit the content relevant to TCR, the WGTP sought to "establish the differences between a technical product and a technological product for the purpose of evaluating program production” (^{Brasil, 2019, p. 22}; translation ours). To do so, they used criteria such as social impact, applicability, innovation, and complexity. It is worth noting the discriminatory potential attributed to the criterion of "innovation": "A product derived from the adaptation of existing knowledge will be considered a technical product and not technological" (^{Brasil, 2019, p. 2}; translation ours). Also noteworthy is the definition assigned to technological product:

A technological product is a 'tangible object' with a high degree of novelty resulting from the application of new scientific knowledge, techniques, and expertise developed in the context of research in graduate programs, used directly to solve problems of companies producing goods or providing services to the population for the benefit of society (^{Brasil, 2019, p. 22}; translation ours).

In both cases, the definition of a technical product and a technological product, it is possible to perceive a theoretical gap concerning the definitions used, which equate expressions like "innovation" and "novelty" and use the degree of innovation (incremental versus radical) to differentiate technical from technological products, without a strong theoretical basis, something unsupported by the literature in the field (^{Wolfe, 1994}; ^{Gregor; Hevner, 2013}; ^{Plonski, 2017}; Lu; Matui; Gracioso, 2019). In the same line of reasoning, the WGTP presents criteria for evaluating the "innovation" criterion:

•Production with a high level of innovation: Development based on unpublished knowledge; •Production with a medium level of innovation: Combination of pre-established knowledge; •Production with a low level of innovation: Adaptation of existing knowledge; •Production with no apparent innovation: Technical production (^{Brasil, 2019, p. 24}; translation ours).

These criteria have problems with their conceptual boundaries, as it becomes impractical to distinguish "Combination" from "Adaptation," as well as "pre-established knowledge" from "existing knowledge," or even to accurately and objectively distinguish high, medium, and low levels of innovation. Criteria, as analytical categories, need to have well-established boundaries in order to allow for the proper delineation of the object of evaluation without ambiguities (^{Laville; Dionne, 1999}; ^{Moraes; Galiazzi, 2006}; ^{Mamede, 2016}; ^{Carlomagno; Rocha, 2016}).

In this regard, we may find an alternative to the current classification in ^{Gregor and Hevner (2013)}, where they propose that invention corresponds to new solutions to new problems, improvement represents a new solution to a known problem, exaptation⁶ relates to a new use of known solutions for new problems, and application⁷ means the use of known solutions in known problems⁸, which blurs innovation within the concepts of improvement and exaptation. For ^{Martens and Pedron (2019)} and ^{Martens, Pedron, and Oliveira (2021)}, of these four categories, only the first three relate to research problems at the stricto sensu level, as they require a systematic review of scientific and technical literature or even gray literature, and therefore have the potential to be included in a technical report, as they understand it.

In addition to the conceptual weakness of the criteria provided by the WGTP (^{Brasil, 2019}), there is a contradiction in the statement that "A production with no apparent innovation" constitutes a "Technical production" when, in a previous section, it was stated that "A product derived from the adaptation of existing knowledge will be considered a technical product and not technological" (^{Brasil, 2019, p. 2}; translation ours), as another example of argumentative and theoretical weakness. Such characteristics in a report compromise its clarity and usefulness, limiting its ability to convince the reader, as required by this type of work, as warned by ^{Pereira (2012)}, ^{Porto and Gurgel (2018)}, and ^{Souza (2022)}.

To conclude, the usefulness of the results is an element that directly derives from the validity of the conclusions, which is dependent on theoretical precision and argumentative coherence, both of which are absent in most of the text, compromising validity and, consequently, usefulness. The validity of an instrument is related to its ability to measure and exactly delimit what it proposes to measure (^{Martins, 2006}; ^{Souza; Alexandre; Guirardello, 2017}), which is reflected in the validity of the conclusions drawn from it. If there are inconsistencies in the conceptual boundaries of the criteria or categories proposed as references for analysis, a portion of the object under study may overlap with another analytical cut of the same object, creating noise in the interpretation of the other. Thus, an evaluation instrument with such characteristics will be a source of bias for the conclusions derived from its use in the analysis of other objects. Given the findings listed so far, the final considerations follow.

4 Final considerations

We understand that a technical-scientific report and a technical-pedagogical report - even though there may be areas of intersection between them, producing a hybrid type, as the TCR seems to be - differ from each other in terms of the origin of the content (the first, systematic research; the second, the account of an academic experience, which may or may not result from research) and their purpose (the first, the communication of research findings to third parties; the second, the documentation of an activity, for oneself or others, and the personal cognitive enrichment of the author). Within the limits established by the scientific culture within the field in question, both differ little from the structure and content envisaged for a scientific article, and there should be similarity in evaluating the quality of the three genres in terms of criteria such as objectivity, conciseness, clarity, parsimony, precision, organization, usefulness, ethics, technical formalism, theoretical-practical correspondence, argumentative coherence, linguistic-grammatical rigor, and validity of conclusions. In this sense, the main difference between them is the absence, in the first two (technical-scientific report and technical-pedagogical report), of the rigor of peer review, which provides prior qualification and relieves the evaluator of the need to evaluate and relativize the quality of the manuscript according to their own criteria, relying on the credibility of the criteria that have previously endorsed it.

Based on the findings obtained from the analysis of the WGTP Report, we conclude that it does not possess the necessary characteristics to serve as a reliable reference for the evaluation of technical reports, given the numerous identified flaws that can be summarized as follows:

Furthermore, the findings demonstrate that the WGTP Report does not allow for a proper understanding of its theoretical assumptions and how to effectively and objectively apply its guidelines, as it unintentionally and excessively delegates to its users - according to subjective criteria of their own and potentially divergent from the desired references by the creators of the WGTP Report - the filling of the theoretical, conceptual, and methodological gaps it intended to present in its guidelines.

Given the widespread flaws identified in the structure and content of the object studied here, this study raises suspicions that the other products presented within the WGTP Report may also have similar issues to those identified here for the TCR. We believe that there is ample room for further empirical and specific studies on the broader theme of this work, in order to establish precise quality criteria and indicators capable of guiding the task of evaluating reports, as a gray literature element capable of effectively contributing to the dissemination of knowledge in a secure, clear, and useful manner.