Slowly progressing and complex health conditions, such as Systemic Arterial Hypertension (SAH), Diabetes Mellitus (DM), and obesity, whose etiology involves genetic, behavioral, environmental, and social factors, are considered major challenges to global public health, leading to gradual complications and significantly impacting the lives of individuals and their families [1].

Collectively, these chronic conditions and their complications account for more than half of the deaths occurring in Brazil [2]. In 2019, according to the Pesquisa Nacional de Saúde (PNS, National Survey of Health) [3], among individuals over 18 years of age, 23.9% reported a diagnosis of SAH and 7.7% of DM. Regarding excess weight, the global estimate is that approximately 4 billion people will be affected worldwide by 2035, representing a significant increase compared to the prevalence recorded in 2020, which was 2.6 billion [4].

In Brazil, the management of chronic conditions is primarily conducted through the efforts of health teams working in the Primary Health Care (PHC), the entry point for users into the Sistema Único de Saúde (SUS, Unified Health System) [5]. Because of its complexity and significant impact on people’s lives, the work of health teams should include not only the clinical aspects of the disease but also the biopsychosocial and environmental perspective through intersectoral actions that also consider the family and community context [6].

Although care for chronic conditions has advanced in promoting healthy lifestyle habits, the lack of intersectoral actions, the disarticulation of teams, the poor infrastructure of service sites, and the scarce number of training actions compromise the quality of care and have not proven sufficient to contain their progression [5,7]. Therefore, in line with the recommendations of the World Health Organization, it is essential to maintain a continuous and systematic evaluation of work processes [7]. This monitoring, combined with educational initiatives and critical reflection, aims to support planning and decision-making, with the goal of ensuring the provision of high-quality care within the scope of health services [1,6].

The self-assessment approach emerges as a crucial element in this context. By proposing a reflective analysis of the practices and work processes of health teams, self-assessment aims to provide an internal and critical view of the activities performed, in addition to providing a space for health professionals to recognize their own contributions and challenges faced in patient care, contributing to the empowerment and autonomy of health professionals [8,9].

However, despite efforts to develop instruments for evaluating the practices of health teams in Brazil, notably the Primary Care Assessment Tool (PCATool) [10] and the Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ-AB, National Program for Improving Access and Quality in Primary Care) [11], there are still limitations, especially regarding specific health care for chronic conditions and the assessment of the teams’ insertion in the territory [12].

Thus, this study aimed to construct and validate a self-assessment instrument for the process and work and management practices related to the care of obesity, SAH, DM, smoking, and their associated risk factors, intended for health care teams working with chronic health conditions in the State of São Paulo.

This is a methodological study following, in a systematic process, the guidelines recommended by Coluci et al. [13] and adopting the Nominal Group Technique (NGT) for content validation, according to the recommendations of Rubio et al. [14] and Deslandes et al. [15].

The PHC service self-assessment instrument for health teams in the care of obesity, SAH, DM, and smoking, henceforth referred to as AuditRRAS-APS, was developed as part of the project “Training in Non-Communicable Chronic Diseases and their associated risk factors in the State of São Paulo”, called ArticulaRRAS. Its application covered both the diagnostic and formative phases of the participating teams.

The choice of the NGT, also known as an expert panel for content validation, was based on its participatory and consensual approach. The technique involves conducting collective meetings during which participants record their opinions in writing, followed by group discussions to explore emerging topics and themes in depth. This participatory approach offers the advantage of integrating different expert perspectives in the field and allows for consensus in a faster and more transparent manner [15]. Furthermore, the use of NGT in an online format has been identified as an efficient strategy [16].

The development of AuditRRAS-APS was conducted in 5 stages (Chart 1): 1) Establishment of the conceptual framework; 2) Construction of items and response scales; 3) Selection and organization of items; 4) Structuring the initial version of the instrument; and 5) Content validation.

Establishment of the conceptual framework: The development of the self-assessment instrument, considering new objectives and target population, was based on a previous questionnaire evaluating care provided to people with excess weight and obesity within the scope of PHC in Brazil [17], organized into 4 dimensions: 1) Organization of Comprehensive Care in Primary Health Care, 2) Organization of Primary Health Care, 3) Institutionalization of public policies for obesity in Primary Health Care, and 4) Health System Performance.

Construction of items and response scales: After defining the conceptual framework, the construction of items and response scales began, based on an exploratory literature review and official Ministry of Health documents to incorporate specific guidelines. The elaboration of the first version of the instrument was carried out collaboratively in meetings with three main researchers. An expanded group of experts linked to the ArticulaRRAS project, working in the areas of endocrinology, nursing, nutrition, physical activity, and public management with PHC work experience, referred to here as internal experts, assisted in decision-making and the content to be included. Additionally, an advisor specialized in public policy network analysis was also consulted.

Selection and organization of items: Based on consultations with experts, the main researchers organized the instrument items into their respective criteria and dimensions, which were subsequently submitted to a second consultation with internal experts.

Structuring the initial version of the instrument: Through a careful review of the clarity and organization of items and criteria, the instrument was diagrammed, and subsequently, the NGT guiding script was developed.

Content validation: The NGT involved five strategic moments. These moments included initial clarifications to experts, silent classification of items (Dimensions; Domains; Items), synthesis of evaluations, collective discussions, and finally, final discussions to reach consensus on the evaluated items and necessary changes. Those involved in this stage were called ad hoc experts and were invited through indications from internal experts and/or PHC management at the state and federal levels. The undergraduate studies field and the most recent degree are presented in Chart 2.

A second round of the NGT was necessary to consolidate consensus among ad hoc experts. The results of this round were integrated into the instrument after a comprehensive and final discussion.

After instrument validation, a Filling Guidance Manual was developed in order to clarify doubts and offer essential guidelines for teams during the discussion process. Additionally, an online version of AuditRRAS-APS was developed with the support of a consultant specializing in digital tools.

This study was approved by the Research Ethics Committee of the Faculdade de Medicina de Botucatu (UNESP-FMB, Medical School of São Paulo State University) (opinion number 5.178.658 and Certificate of Presentation of Ethical Appreciation - CAAE: 52619821.3.0000.5411).

The AuditRRAS-APS development and validation process lasted from March to October 2021 and involved, in addition to the three main researchers, 20 experts (five internal and 14 ad hoc) and an advisor in public policy networks.

Of the 14 ad hoc experts, six participated in both rounds, while four were present only in the first round and four participated only in the second round.

The initial version of the instrument consisted of 180 items organized into 28 criteria distributed across the 4 dimensions. Experts expressed concern about the length of the instrument. However, after the suggested inclusions and exclusions, there was a reduction of only seven items, while most underwent only semantic and grammatical changes.

Furthermore, initially, the proposal was for the instrument to be completed by the professional responsible for the unit. However, after discussions with experts, the importance of the entire team responding to the instrument was recognized. This modification aimed to enable collective discussion and increase the likelihood that the instrument would adequately reflect the reality of the service.

The original criteria, excluded, modified, and/or maintained in the instrument, are detailed in Chart 3. Among the general changes, the need for greater emphasis on smoking was emphasized, as well as the adaptation of the use of some terms related to chronic conditions and physical activity practice. Changes were suggested in all dimensions, as detailed below.

In Dimension 1, it was suggested to improve the characterization of territorial resources; improve issues related to intersectorality in the territory, food insecurity, perspectives on the post-COVID-19 pandemic, and the approach to self-care. For Dimension 2, greater alignment with the systems and programs in force in the SUS and the inclusion of more items related to intersectorality in management were suggested. Additionally, the need for this dimension to be answered not only by the team but also by municipal management was identified. In Dimension 3, the need to incorporate information on how official documents are received and used was emphasized, along with the inclusion of a series of recent documents. Within the scope of Dimension 3, there were items related to social participation, and the need to separate them into a fifth dimension was perceived. Finally, in Dimension 4, the need for greater clarity regarding its objectives was pointed out. Additionally, due to the COVID-19 pandemic context, in which many teams were still in the process of resuming face-to-face activities, it was suggested that this dimension be directed to assess the period before the start of the pandemic.

Thus, AuditRRAS-APS, validated by experts, consisted of 173 items, organized into 29 criteria and distributed across five dimensions. These dimensions include: 1) Organization of Comprehensive Care in Primary Health Care; 2) Organization and management of Primary Health Care; 3) Institutionalization of public policies for obesity, DM, SAH and smoking and their risk factors in PHC; 4) Social participation; and 5) Self-perception of the performance of the Brazilian Unified Health System. The complete version of the instrument is available at the following link (https://articularras.com.br/wp-content/uploads/2024/01/Instrumento-Painel-de-Especialista-ArticulaRRAS-281021.pdf).

Consultation with the digital tools specialist involved three meetings, during which the best strategies for optimizing the tool were discussed. One of the main contributions of this stage was the choice of the Google Forms tool. Additionally, adjustments were made to the presentation text and guidelines on its online completion. Following the specialist’s guidance, the online version was subdivided into five distinct forms, aiming to facilitate completion and minimize dropouts, allowing each form to be answered independently.

The development and validation of AuditRRAS-APS resulted in an innovative tool for PHC, adaptable and sensitive to the needs of teams dealing with the complexities of chronic conditions. This instrument has the potential not only to improve the quality of care but also to strengthen the capacity for self-reflection and continuous improvement by health professionals, contributing to the effectiveness and efficiency of the health system as a whole.

The construction of the instrument followed a robust methodological approach, utilizing the Nominal Group Technique and involving internal and ad hoc experts. The collective discussion, guided by the NGT, is an important ally in identifying weaknesses and decision-making in the construction and validation of assessment instruments [16]. By involving different professionals from various backgrounds and areas of expertise, as well as from distinct geographical locations, facilitated by the online modality, the technique allowed for a comprehensive perspective on the object of study and ensured the equitable weighting of expert opinions [18].

The importance of conducting extensive consultation during the instrument’s construction is highlighted, involving literature review and expert consultation [13,15].

This inclusive approach aims to ensure a solid theoretical foundation and practical applicability. The fact that it is not restricted to a strictly quantitative evaluation allows experts to freely express and discuss their opinions, enabling critiques, suggestions, and clarification of scores. This facilitates the understanding of the evaluator’s perspective and makes the instrument’s evaluation more complete and enriching [19].

The evaluation of health services plays a fundamental role in strengthening and structuring the health system, allowing for adequate planning to meet the local and individual demands of the teams [20]. This practice is systematically and recurrently recommended by numerous global forums [7; thus, including self-assessment as an initial part of a formative process can be an important tool for professional engagement and awareness.

Health evaluation, as highlighted by Alberto et al. [21], requires a collective construction that provides agreement and self-reflection, involving various professional categories. Engaging different professional categories, as proposed by AuditRRAS-APS, in decision-making and diagnosis, decentralizing these functions, can result in a sense of belonging that stimulates the autonomy of members. This team interaction and decentralization of health decisions, often exclusive to nursing, contributes to the improvement of service quality, as indicated by Marchetti et al. [6] and Santos et al. [8].

In the context of PHC, where team interactions and performance are permeated by subjectivity and complexity, a critical process of knowledge and practice construction and reconstruction is essential [6]. This approach not only benefits the care of chronic conditions but also contributes to the overall improvement of the health system’s quality, promoting the reduction of complication severity, hospitalization time, and costs related to avoidable hospitalizations [7,22].

Team meeting and discussion moments stand out as powerful opportunities for work planning and organization, with teams that incorporate self-assessment processes demonstrating better performance [20]. However, various reasons are cited as justification for the non-holding of team meetings, such as team distancing, work overload, physical space limitations, vertical interventions, low professional retention, lack of continuing education, and the valorization of the number of consultations over quality [5-7]. An instrument that guides the self-assessment process and can be used periodically by the team, either in its entirety or considering selected dimensions, can stimulate this reflection, in addition to producing important evaluation indicators.

Furthermore, in a study conducted by Santos et al. [8], unit managers often blame other professionals for not being familiar with the principles and guidelines of the PHC. Thus, the change suggested by experts in AuditRRAS-APS, transforming discussions into participatory and collaborative team responses, can make the instrument a valuable tool for appropriating still underutilized discussion moments.

Despite various attempts to develop self-assessment instruments for teams, this is still a gap to be overcome. The Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ-AB, National Program for Improving Access and Quality in Primary Care) [11], which involved the participation of civil society and academic institutions in its external evaluation, stands out as one of the most successful experiences; however, it was discontinued in 2019, after completing its third cycle [12,20].

In 2019, the Programa Previne Brasil (PPB, Prevent Brazil Programme) [23] was created, representing a setback by focusing the evaluation on a limited number of goals, neglecting the territory and the community [12]. Additionally, even previously established programs, such as PMAQ, and other validated instruments, faced challenges in comprehensively evaluating grouped chronic conditions, especially SAH, DM, and obesity. These difficulties, as discussed by Neves et al. [20] and Lopes et al. [24], are largely attributed to the strictly quantitative approach of these instruments, which may not fully capture the complexity involved in these health conditions.

Furthermore, one of the main deficiencies observed in the use of existing instruments is the lack of effective integration of organizational processes and professional practices within the territory. This results in a limited view of care, which fails to adequately break with the biomedical and hegemonic model [8,12].

According to Borges et al. [22], the socioeconomic characteristics of the population and the environment play a significant role in the demands faced by health teams. Having in-depth knowledge of the territory and discussing this data are crucial steps for reducing avoidable hospitalizations. In this context, the inclusion in AuditRRAS-APS of questions related to articulated work between the various points of the network, combined with the diagnosis of this reality, acts as a valuable stimulus for the teams.

Additionally, addressing chronic conditions requires articulation of other health promotion policies and the adoption of an intersectoral approach. It is essential to create partnerships and involve other services and institutions, whether public or private, such as schools, industries, businesses, leisure spaces, and religious institutions. This approach means advancing towards articulation beyond the physical space of the unit and the exclusive scope of the health sector [21]. Integrating these resources into the AuditRRAS-APS instrument can be crucial to encourage teams to strengthen partnerships, providing health professionals with a broader view of the various possibilities involved. In this regard, the AuditRRAS-APS has innovated by incorporating elements of the territory, public policy networks, and social participation in team reflections, aiming to inspire teams toward a more comprehensive and expanded practice.

AuditRRAS-APS was developed to include all the technical guidelines from the official documents for chronic conditions, even knowing that the implementation of these guidelines in practice is far from happening. Regarding this distance between theory and practice, it becomes crucial for health professionals to appropriate the various policies and tools for service quality promotion [8]. The instrument’s responses, constructed in a multiprofessional manner and based on common experience, in addition to stimulating critical-reflective development, have the potential to enrich and align the performance of health teams with the principles of the SUS [6-8].

The main disadvantage of the AuditRRAS-APS instrument is its size. Producing a comprehensive and, at the same time, compact instrument presents a significant challenge. To make it more functional and useful, it is suggested, as guided in the filling manual developed, the fractionation of discussions by dimension. Additionally, in response to the identified needs, using each dimension individually can be an effective strategy.

Among the study’s weaknesses, the lack of anonymity in the NGT stands out, which may influence the more critical positioning of participants when expressing their opinions, especially if they are linked to specific organizations or institutions. To circumvent this situation, the strategy adopted included the use of individual forms before the collective discussions of the panels with ad hoc experts. This approach allowed participants to express themselves more comfortably and without influence from the collective discussion, contributing to the quality and authenticity of the data collected during the study.

The AuditRRAS-APS constituted an innovative tool, developed and validated through rigorous methodology, to be used in the self-assessment of the work and management process and practices related to the care of obesity, SAH, DM, smoking, and their associated risk factors, intended for health care teams working with chronic health conditions in the State of São Paulo. It can be used both in research and in service routines, with self-assessment purposes. Thus, it is concluded that despite its length, the AuditRRAS-APS was well evaluated, demonstrating that the approach employed in its elaboration was comprehensive enough to meet the expectations of the experts.