<?xml version="1.0" encoding="ISO-8859-1"?><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<front>
<journal-meta>
<journal-id>0103-6831</journal-id>
<journal-title><![CDATA[Estudos em Avaliação Educacional]]></journal-title>
<abbrev-journal-title><![CDATA[Est. Aval. Educ.]]></abbrev-journal-title>
<issn>0103-6831</issn>
<publisher>
<publisher-name><![CDATA[Fundação Carlos Chagas]]></publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id>S0103-68312022000100227</article-id>
<article-id pub-id-type="doi">10.18222/eae.v33.8938</article-id>
<title-group>
<article-title xml:lang="pt"><![CDATA[APRIMORAMENTO DO CT PUZZLE TEST PARA AVALIAÇÃO DO PENSAMENTO COMPUTACIONAL]]></article-title>
<article-title xml:lang="es"><![CDATA[MEJORA DEL CT PUZZLE TEST PARA EVALUAR EL PENSAMIENTO COMPUTACIONAL]]></article-title>
<article-title xml:lang="en"><![CDATA[IMPROVEMENT OF THE CT PUZZLE TEST FOR COMPUTATIONAL THINKING ASSESSMENT]]></article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname><![CDATA[CALBUSCH]]></surname>
<given-names><![CDATA[LEONARDO FELIPE DE ÁVILA]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname><![CDATA[COUTO]]></surname>
<given-names><![CDATA[NATÁLIA ELLERY RIBEIRO]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname><![CDATA[ROCCA]]></surname>
<given-names><![CDATA[JULIA ZANETTI]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname><![CDATA[RAABE]]></surname>
<given-names><![CDATA[ANDRÉ LUÍS ALICE]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
</contrib-group>
<aff id="Af1">
<institution><![CDATA[,Universidade do Vale do Itajaí Instituto Federal Catarinense ]]></institution>
<addr-line><![CDATA[Brusque SC]]></addr-line>
<country>Brasil</country>
</aff>
<aff id="Af2">
<institution><![CDATA[,Universidade do Vale do Itajaí  ]]></institution>
<addr-line><![CDATA[Itajaí SC]]></addr-line>
<country>Brasil</country>
</aff>
<aff id="Af3">
<institution><![CDATA[,Universidade Federal de Rondonópolis  ]]></institution>
<addr-line><![CDATA[Rondonópolis MT]]></addr-line>
<country>Brasil</country>
</aff>
<aff id="Af4">
<institution><![CDATA[,Universidade do Vale do Itajaí  ]]></institution>
<addr-line><![CDATA[Itajaí SC]]></addr-line>
<country>Brasil</country>
</aff>
<pub-date pub-type="pub">
<day>00</day>
<month>00</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="epub">
<day>00</day>
<month>00</month>
<year>2022</year>
</pub-date>
<volume>33</volume>
<copyright-statement/>
<copyright-year/>
<self-uri xlink:href="http://educa.fcc.org.br/scielo.php?script=sci_arttext&amp;pid=S0103-68312022000100227&amp;lng=en&amp;nrm=iso"></self-uri><self-uri xlink:href="http://educa.fcc.org.br/scielo.php?script=sci_abstract&amp;pid=S0103-68312022000100227&amp;lng=en&amp;nrm=iso"></self-uri><self-uri xlink:href="http://educa.fcc.org.br/scielo.php?script=sci_pdf&amp;pid=S0103-68312022000100227&amp;lng=en&amp;nrm=iso"></self-uri><abstract abstract-type="short" xml:lang="pt"><p><![CDATA[RESUMO O pensamento computacional é uma competência ligada à resolução de problemas que mobiliza o pensamento algorítmico, a identificação de padrões, a abstração e a modula- rização de problemas e suas soluções. Esse tema tem sido incorporado em currículos escolares de vários países, incluindo o Brasil, mas são poucos os instrumentos de mensuração validados. Este artigo trata da busca de evidências de validade do CT Puzzle Test, um instrumento de diagnóstico do pensamento computacional baseado em puzzles. Foi avaliada a validade convergente com um teste que já tem evidências de validade estabelecidas. Os resultados confirmam a convergência entre os instrumentos. Por fim, foi aprimorado o cálculo do escore do CT Puzzle Test e evidenciada a validade do uso de puzzles como instrumento de avaliação.]]></p></abstract>
<abstract abstract-type="short" xml:lang="es"><p><![CDATA[RESUMEN El pensamiento computacional es una competencia conectada a la resolución de problemas que moviliza el pensamiento algorítmico, la identificación de estándares, la abstracción y la modularización de problemas y sus soluciones. Este tema ha sido incorporado a currículos escolares de varios países, incluso de Brasil, pero hay pocos instrumentos válidos para medirlo. Este artículo trata de la búsqueda de evidencias de validez del CT Puzzle Test, un instrumento de diagnóstico del pensamiento computacional basado en puzzles. La validez convergente se evaluó con otro test que ya ha establecido evidencia de validez. Los resultados confirman la convergencia entre los instrumentos. Además, se mejoró el cálculo del escore del CT Puzzle Test y se evidenció la validez del uso de puzzles como instrumento de evaluación.]]></p></abstract>
<abstract abstract-type="short" xml:lang="en"><p><![CDATA[ABSTRACT Computational thinking is a problem-solving skill that involves algorithmic thinking, pattern identification, abstraction, and the modularization of problems and their solutions. This subject has been incorporated into school curricula in several countries, including Brazil, but there are few instruments validated to measure it. The present article deals with searches for evidence of the validity of the CT Puzzle Test, a puzzle-based tool to diagnose computational thinking. Convergent validity was verified with another test that already has the evidence of its validity established. The results confirm the convergence between the instruments. In addition, the score calculation for the CT Puzzle Test items was improved and the validity of adopting puzzles as an assessment tool was evidenced.]]></p></abstract>
<kwd-group>
<kwd lng="pt"><![CDATA[RESOLUÇÃO DE PROBLEMAS]]></kwd>
<kwd lng="pt"><![CDATA[AVALIAÇÃO DA APRENDIZAGEM]]></kwd>
<kwd lng="pt"><![CDATA[INSTRUMENTOS DE AVALIAÇÃO]]></kwd>
<kwd lng="pt"><![CDATA[VALIDADE DE TESTES.]]></kwd>
<kwd lng="es"><![CDATA[RESOLUCIÓN DE PROBLEMAS]]></kwd>
<kwd lng="es"><![CDATA[EVALUACIÓN DEL APRENDIZAJE]]></kwd>
<kwd lng="es"><![CDATA[INSTRUMENTOS DE EVALUACIÓN]]></kwd>
<kwd lng="es"><![CDATA[VALIDEZ DE TESTS.]]></kwd>
<kwd lng="en"><![CDATA[PROBLEM-SOLVING]]></kwd>
<kwd lng="en"><![CDATA[LEARNING ASSESSMENT]]></kwd>
<kwd lng="en"><![CDATA[ASSESSMENT INSTRUMENTS]]></kwd>
<kwd lng="en"><![CDATA[TEST VALIDITY.]]></kwd>
</kwd-group>
</article-meta>
</front><back>
<ref-list>
<ref id="B1">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ARAUJO]]></surname>
<given-names><![CDATA[N. de]]></given-names>
</name>
<name>
<surname><![CDATA[PRIMO]]></surname>
<given-names><![CDATA[T. T.]]></given-names>
</name>
<name>
<surname><![CDATA[PERNAS]]></surname>
<given-names><![CDATA[A. M]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[OntoScratch: ontologias para a avaliação do ensino de Pensamento Computacional através do Scratch]]></article-title>
<source><![CDATA[In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 31., 2020, on-line. Anais [...]]]></source>
<year>2020</year>
<page-range>1823-32</page-range><publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
<publisher-name><![CDATA[Sociedade Brasileira de Computação]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B2">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[BALL]]></surname>
<given-names><![CDATA[M. A.]]></given-names>
</name>
<name>
<surname><![CDATA[GARCIA]]></surname>
<given-names><![CDATA[D. D]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Autograding and feedback for snap!]]></article-title>
<source><![CDATA[In: TECHNICAL SYMPOSIUM ON COMPUTING SCIENCE EDUCATION, 47., 2016, Memphis. Proceedings [&#8230;]]]></source>
<year>2016</year>
<publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B3">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[BITTENCOURT]]></surname>
<given-names><![CDATA[H. R.]]></given-names>
</name>
<name>
<surname><![CDATA[CREUTZBERG]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[RODRIGUES]]></surname>
<given-names><![CDATA[A. C. de M.]]></given-names>
</name>
<name>
<surname><![CDATA[CASARTELLI]]></surname>
<given-names><![CDATA[A. de O.]]></given-names>
</name>
<name>
<surname><![CDATA[FREITAS]]></surname>
<given-names><![CDATA[A. L. S de.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Desenvolvimento e validação de um instrumento para avaliação de disciplinas na educação superior]]></article-title>
<source><![CDATA[Estudos em Avaliação Educacional]]></source>
<year>2011</year>
<volume>22</volume>
<numero>48</numero>
<issue>48</issue>
<page-range>91-113</page-range><publisher-loc><![CDATA[São Paulo ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B4">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[BOCCONI]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
<name>
<surname><![CDATA[CHIOCCARIELLO]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[DETTORI]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
<name>
<surname><![CDATA[FERRARI]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[ENGELHARDT]]></surname>
<given-names><![CDATA[K.]]></given-names>
</name>
<name>
<surname><![CDATA[KAMPYLIS]]></surname>
<given-names><![CDATA[P.]]></given-names>
</name>
<name>
<surname><![CDATA[PUNIE]]></surname>
<given-names><![CDATA[Y.]]></given-names>
</name>
</person-group>
<source><![CDATA[Developing computational thinking in compulsory education: implications for policy and practice]]></source>
<year>2016</year>
<publisher-loc><![CDATA[Luxembourg ]]></publisher-loc>
<publisher-name><![CDATA[Publications Office of the European Union]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B5">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[BRACKMANN]]></surname>
<given-names><![CDATA[C. P.]]></given-names>
</name>
</person-group>
<source><![CDATA[Desenvolvimento do pensamento computacional através de atividades desplugadas na educação básica]]></source>
<year>2017</year>
<publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
<publisher-name><![CDATA[Centro Interdisciplinar de Novas Tecnologias na Educação, Universidade Federal do Rio Grande do Sul]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B6">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[BRACKMANN]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
<name>
<surname><![CDATA[BOUCINHA]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[BARONE]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
<name>
<surname><![CDATA[CASALI]]></surname>
<given-names><![CDATA[A]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Pensamento computacional desplugado: ensino e avaliação na educação primária espanhola]]></article-title>
<source><![CDATA[In: CONGRESSO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 6., 2017, Recife. Anais [...]]]></source>
<year>2017</year>
<publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
<publisher-name><![CDATA[Sociedade Brasileira de Computação]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B7">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[BRENNAN]]></surname>
<given-names><![CDATA[K.]]></given-names>
</name>
<name>
<surname><![CDATA[RESNICK]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
</person-group>
<source><![CDATA[New frameworks for studying and assessing the development of computational thinking]]></source>
<year>2012</year>
<publisher-loc><![CDATA[Vancouver ]]></publisher-loc>
<publisher-name><![CDATA[American Educational Research Association]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B8">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[CORDENONZI]]></surname>
<given-names><![CDATA[W. H.]]></given-names>
</name>
<name>
<surname><![CDATA[DEL PINO]]></surname>
<given-names><![CDATA[J. C.]]></given-names>
</name>
<name>
<surname><![CDATA[CARDOSO]]></surname>
<given-names><![CDATA[V. M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Analisando o desenvolvimento do Pensamento Computacional na disciplina Matemática Discreta]]></article-title>
<source><![CDATA[Revista Brasileira de Informática na Educação]]></source>
<year>2021</year>
<volume>29</volume>
<page-range>880-902</page-range><publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B9">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[COUTO]]></surname>
<given-names><![CDATA[N. E. R.]]></given-names>
</name>
</person-group>
<source><![CDATA[Impacto da implantação de uma disciplina de pensamento computacional em estudantes]]></source>
<year>2018</year>
<publisher-loc><![CDATA[Itajaí, SC ]]></publisher-loc>
<publisher-name><![CDATA[Universidade do Vale de Itajaí]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B10">
<nlm-citation citation-type="book">
<collab>CSTA</collab>
<source><![CDATA[K-12 Computer Science Standards]]></source>
<year>2011</year>
<publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B11">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[DAGIENE]]></surname>
<given-names><![CDATA[V.]]></given-names>
</name>
<name>
<surname><![CDATA[FUTSCHEK]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Bebras international contest on informatics and computer literacy: criteria for good tasks]]></article-title>
<source><![CDATA[Lecture notes in computer science]]></source>
<year>2008</year>
<page-range>19-30</page-range><publisher-loc><![CDATA[S. l ]]></publisher-loc>
<publisher-name><![CDATA[Springer Berlin Heidelberg]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B12">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[FALKNER]]></surname>
<given-names><![CDATA[N.]]></given-names>
</name>
<name>
<surname><![CDATA[SOORIAMURTHI]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
<name>
<surname><![CDATA[MICHALEWICZ]]></surname>
<given-names><![CDATA[Z.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Puzzle-based learning for engineering and computer science]]></article-title>
<source><![CDATA[Computer]]></source>
<year>2010</year>
<volume>43</volume>
<numero>4</numero>
<issue>4</issue>
<page-range>20-8</page-range><publisher-loc><![CDATA[S. l ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B13">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[FEUERSTEIN]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
</person-group>
<source><![CDATA[The Feuerstein Instrumental Enrichment Program]]></source>
<year>2006</year>
<publisher-loc><![CDATA[Jerusalem ]]></publisher-loc>
<publisher-name><![CDATA[ICELP Publications]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B14">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[FRANÇA]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
<name>
<surname><![CDATA[SILVA]]></surname>
<given-names><![CDATA[C. G]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Identificação de critérios para avaliação do pensamento computacional aplicado]]></article-title>
<source><![CDATA[In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 31., 2020, on-line. Anais [...]]]></source>
<year>2020</year>
<page-range>1493-502</page-range><publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
<publisher-name><![CDATA[Sociedade Brasileira de Computação]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B15">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[GONÇALVES]]></surname>
<given-names><![CDATA[F. A.]]></given-names>
</name>
</person-group>
<source><![CDATA[Um instrumento para diagnóstico do pensamento computacional]]></source>
<year>2015</year>
<publisher-loc><![CDATA[Itajaí, SC ]]></publisher-loc>
<publisher-name><![CDATA[Universidade do Vale de Itajaí]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B16">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[GROVER]]></surname>
<given-names><![CDATA[S]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Systems of assessments for deeper learning of computational thinking in k-12]]></article-title>
<source><![CDATA[In: ANNUAL MEETING OF THE AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, 2015, Chicago, IL. Proceedings [&#8230;]]]></source>
<year>2015</year>
<publisher-loc><![CDATA[Washington, DC ]]></publisher-loc>
<publisher-name><![CDATA[American Educational Research Association]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B17">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[HINKLE]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
<name>
<surname><![CDATA[WIERSMA]]></surname>
<given-names><![CDATA[W.]]></given-names>
</name>
<name>
<surname><![CDATA[JURS]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
</person-group>
<source><![CDATA[Applied statistics for the behavioral sciences]]></source>
<year>2003</year>
<publisher-loc><![CDATA[Boston, MA ]]></publisher-loc>
<publisher-name><![CDATA[Houghton Mifflin]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B18">
<nlm-citation citation-type="book">
<collab>INTERNATIONAL SOCIETY FOR TECHNOLOGY IN EDUCATION - ISTE; COMPUTER SCIENCE TEACHERS ASSOCIATION - CSTA</collab>
<source><![CDATA[Computational thinking in k-12 teachers resources]]></source>
<year>2011</year>
<edition>2nd</edition>
<publisher-loc><![CDATA[S. l ]]></publisher-loc>
<publisher-name><![CDATA[ISTE: CSTA]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B19">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[KALELIOGLU]]></surname>
<given-names><![CDATA[F.]]></given-names>
</name>
<name>
<surname><![CDATA[GÜLBAHAR]]></surname>
<given-names><![CDATA[Y.]]></given-names>
</name>
<name>
<surname><![CDATA[KUKUL]]></surname>
<given-names><![CDATA[V.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[A framework for computational thinking based on a systematic research review]]></article-title>
<source><![CDATA[Baltic Journal of Modern Computing]]></source>
<year>2016</year>
<volume>4</volume>
<numero>3</numero>
<issue>3</issue>
<page-range>583-96</page-range><publisher-loc><![CDATA[Riga ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B20">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[KONG]]></surname>
<given-names><![CDATA[S. C.]]></given-names>
</name>
<name>
<surname><![CDATA[ABELSON]]></surname>
<given-names><![CDATA[H.]]></given-names>
</name>
<name>
<surname><![CDATA[LAI]]></surname>
<given-names><![CDATA[M]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Introduction to computational thinking education]]></article-title>
<person-group person-group-type="editor">
<name>
<surname><![CDATA[KONG]]></surname>
<given-names><![CDATA[S. C.]]></given-names>
</name>
<name>
<surname><![CDATA[ABELSON]]></surname>
<given-names><![CDATA[H.]]></given-names>
</name>
</person-group>
<source><![CDATA[Computational Thinking Education]]></source>
<year>2019</year>
<page-range>1-10</page-range><publisher-loc><![CDATA[S. l ]]></publisher-loc>
<publisher-name><![CDATA[Springer Singapore]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B21">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[LANDIS]]></surname>
<given-names><![CDATA[J. R.]]></given-names>
</name>
<name>
<surname><![CDATA[KOCH]]></surname>
<given-names><![CDATA[G. G.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[The measurement of observer agreement for categorical data]]></article-title>
<source><![CDATA[Biometrics]]></source>
<year>1977</year>
<volume>33</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>159-74</page-range><publisher-loc><![CDATA[Bethesda ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B22">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[MAIORANA]]></surname>
<given-names><![CDATA[F.]]></given-names>
</name>
<name>
<surname><![CDATA[GIORDANO]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
<name>
<surname><![CDATA[MORELLI]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Quizly: a live coding assessment platform for app inventor]]></article-title>
<source><![CDATA[In: IEEE BLOCKS AND BEYOND WORKSHOP, 2015, [S. l.]. Proceedings [&#8230;]]]></source>
<year>2015</year>
<page-range>25-30</page-range><publisher-loc><![CDATA[S. l ]]></publisher-loc>
<publisher-name><![CDATA[IEEE]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B23">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[MERRICK]]></surname>
<given-names><![CDATA[K. E.]]></given-names>
</name>
</person-group>
<source><![CDATA[An empirical evaluation of puzzle-based learning as an interest approach for teaching introductory computer science]]></source>
<year>2010</year>
<publisher-loc><![CDATA[S. l ]]></publisher-loc>
<publisher-name><![CDATA[IEEE Transactions on Education]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B24">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[MICHALEWICZ]]></surname>
<given-names><![CDATA[Z.]]></given-names>
</name>
<name>
<surname><![CDATA[MICHALEWICZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Puzzle-based learning: introduction to critical thinking, Mathematics, and problem solving]]></source>
<year>2008</year>
<publisher-loc><![CDATA[Melbourne ]]></publisher-loc>
<publisher-name><![CDATA[Hybrid Publishers]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B25">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[MORENO-LEÓN]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
<name>
<surname><![CDATA[ROBLES]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M. Dr.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Scratch: automatic analysis of scratch projects to assess and foster computational thinking]]></article-title>
<source><![CDATA[RED-Revista de Educación a Distancia]]></source>
<year>2015</year>
<numero>46</numero>
<issue>46</issue>
<page-range>1-23</page-range><publisher-loc><![CDATA[Murcia ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B26">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[MÜHLING]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[RUF]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[HUBWIESER]]></surname>
<given-names><![CDATA[P]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Design and first results of a psychometric test for measuring basic programming abilities]]></article-title>
<source><![CDATA[In: THE WORKSHOP IN PRIMARY AND SECONDARY COMPUTING EDUCATION, 2015, London. Proceedings [&#8230;]]]></source>
<year>2015</year>
<publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B27">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[OTA]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
<name>
<surname><![CDATA[MORIMOTO]]></surname>
<given-names><![CDATA[Y.]]></given-names>
</name>
<name>
<surname><![CDATA[KATO]]></surname>
<given-names><![CDATA[H]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Ninja code village for scratch: Function samples/function analyser and automatic assessment of computational thinking concepts]]></article-title>
<source><![CDATA[In: IEEE SYMPOSIUM ON VISUAL LANGUAGES AND HUMAN-CENTRIC COMPUTING (VL/HCC), set. 2016, Cambridge, UK. Proceedings [...]]]></source>
<year>2016</year>
<publisher-loc><![CDATA[Cambridge, UK ]]></publisher-loc>
<publisher-name><![CDATA[IEEE]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B28">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[PAPERT]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
</person-group>
<source><![CDATA[Teaching children thinking]]></source>
<year>1971</year>
<publisher-loc><![CDATA[Cambridge, MA ]]></publisher-loc>
<publisher-name><![CDATA[Massachusetts Institute of Technology]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B29">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[PASQUALI]]></surname>
<given-names><![CDATA[L.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Validade dos Testes]]></article-title>
<source><![CDATA[Examen: Política, Gestão e Avaliação da Educação]]></source>
<year>2017</year>
<volume>1</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>36</page-range></nlm-citation>
</ref>
<ref id="B30">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[RAABE]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[SANTANA]]></surname>
<given-names><![CDATA[A. L. M.]]></given-names>
</name>
<name>
<surname><![CDATA[ELLERY]]></surname>
<given-names><![CDATA[N.]]></given-names>
</name>
<name>
<surname><![CDATA[GONÇALVES]]></surname>
<given-names><![CDATA[F]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Um instrumento para diagnóstico do pensamento computacional]]></article-title>
<source><![CDATA[In: CONGRESSO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 6., 2017, Recife. Anais [...]]]></source>
<year>2017</year>
<publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
<publisher-name><![CDATA[Sociedade Brasileira de Computação]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B31">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[RAABE]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[ZORZO]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
<name>
<surname><![CDATA[BLIKSTEIN]]></surname>
<given-names><![CDATA[P.]]></given-names>
</name>
</person-group>
<source><![CDATA[Computação na educação básica: fundamentos e experiências]]></source>
<year>2020</year>
<publisher-loc><![CDATA[Porto Alegre ]]></publisher-loc>
<publisher-name><![CDATA[Penso]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B32">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROBERTS]]></surname>
<given-names><![CDATA[P.]]></given-names>
</name>
<name>
<surname><![CDATA[PRIEST]]></surname>
<given-names><![CDATA[H.]]></given-names>
</name>
<name>
<surname><![CDATA[TRAYNOR]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Reliability and validity in research]]></article-title>
<source><![CDATA[Nursing Standard]]></source>
<year>2006</year>
<volume>20</volume>
<numero>44</numero>
<issue>44</issue>
<page-range>41-5</page-range></nlm-citation>
</ref>
<ref id="B33">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROBLES]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
<name>
<surname><![CDATA[HAUCK]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[NOMBELA ALONSO]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
<name>
<surname><![CDATA[GRESSE VON WANGENHEIM]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
<name>
<surname><![CDATA[MORENO-LEÓN]]></surname>
<given-names><![CDATA[J]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[On tools that support the development of computational thinking skills: Some thoughts and future vision]]></article-title>
<source><![CDATA[In: INTERNATIONAL CONFERENCE ON COMPUTATIONAL THINKING EDUCATION, 2018, Hong Kong. Proceedings [&#8230;]]]></source>
<year>2018</year>
<page-range>125-9</page-range><publisher-loc><![CDATA[Hong Kong ]]></publisher-loc>
<publisher-name><![CDATA[The Education University of Hong Kong]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B34">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[RODRIGUES]]></surname>
<given-names><![CDATA[M. M. M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Proposta de análise de itens das provas do Saeb sob a perspectiva pedagógica e psicométrica]]></article-title>
<source><![CDATA[Estudos em Avaliação Educacional]]></source>
<year>2006</year>
<volume>17</volume>
<numero>34</numero>
<issue>34</issue>
<page-range>43-78</page-range><publisher-loc><![CDATA[São Paulo ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B35">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[RODRIGUEZ]]></surname>
<given-names><![CDATA[C.]]></given-names>
</name>
<name>
<surname><![CDATA[REIS]]></surname>
<given-names><![CDATA[R. C. D.]]></given-names>
</name>
<name>
<surname><![CDATA[ISOTANI]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Recursos e estratégias para desenvolvimento e avaliação do Pensamento Computacional na escola]]></article-title>
<source><![CDATA[Tecnologias, Sociedade e Conhecimento]]></source>
<year>2017</year>
<volume>4</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>59-81</page-range><publisher-loc><![CDATA[Campinas, SP ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B36">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Computational thinking test: Design guidelines and content validation]]></article-title>
<source><![CDATA[In: ANNUAL INTERNATIONAL CONFERENCE ON EDUCATION AND NEW LEARNING TECHNOLOGIES, 7., 2015, Barcelona. Proceedings [&#8230;]]]></source>
<year>2015</year>
<publisher-loc><![CDATA[Barcelona ]]></publisher-loc>
<publisher-name><![CDATA[IATED]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B37">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Codigoalfabetización y pensamiento computacional en educación primaria y secundaria: validación de un instrumento y evaluación de programas]]></source>
<year>2016</year>
<publisher-loc><![CDATA[Madrid ]]></publisher-loc>
<publisher-name><![CDATA[Universidad Nacional de Educación a Distancia]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B38">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[MORENO-LEÓN]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
<name>
<surname><![CDATA[ROBLES]]></surname>
<given-names><![CDATA[G.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Complementary tools for computational thinking assessment]]></article-title>
<source><![CDATA[In: INTERNACIONAL CONFERENCE ON COMPUTATIONAL THINKING EDUCATION, 2017, Hong Kong. Proceedings [&#8230;]]]></source>
<year>2017</year>
<page-range>154-9</page-range><publisher-loc><![CDATA[Hong Kong ]]></publisher-loc>
<publisher-name><![CDATA[The Education University of Hong Kong]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B39">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[MORENO-LEÓN]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
<name>
<surname><![CDATA[ROBLES]]></surname>
<given-names><![CDATA[G]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Combining assessment tools for a comprehensive evaluation of computational thinking interventions]]></article-title>
<person-group person-group-type="editor">
<name>
<surname><![CDATA[KONG]]></surname>
<given-names><![CDATA[S. C.]]></given-names>
</name>
<name>
<surname><![CDATA[Abelson]]></surname>
<given-names><![CDATA[H.]]></given-names>
</name>
</person-group>
<source><![CDATA[Computational Thinking Education]]></source>
<year>2019</year>
<page-range>79-98</page-range><publisher-loc><![CDATA[S. l ]]></publisher-loc>
<publisher-name><![CDATA[Springer Singapore]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B40">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[PÉREZ-GONZÁLEZ]]></surname>
<given-names><![CDATA[J.-C.]]></given-names>
</name>
<name>
<surname><![CDATA[JIMÉNEZ-FERNÁNDEZ]]></surname>
<given-names><![CDATA[C]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Which cognitive abilities underlie computational thinking?]]></article-title>
<source><![CDATA[Criterion validity of the computational thinking test. Computers in Human Behavior]]></source>
<year>2017</year>
<volume>72</volume>
<page-range>678-91</page-range><publisher-loc><![CDATA[Amsterdam ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B41">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ROMÁN-GONZÁLEZ]]></surname>
<given-names><![CDATA[M.]]></given-names>
</name>
<name>
<surname><![CDATA[PÉREZ-GONZÁLEZ]]></surname>
<given-names><![CDATA[J.-C.]]></given-names>
</name>
<name>
<surname><![CDATA[MORENO-LEÓN]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
<name>
<surname><![CDATA[ROBLES]]></surname>
<given-names><![CDATA[G]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Can computational talent be detected?]]></article-title>
<source><![CDATA[Predictive validity of the Computational Thinking Test. International Journal of Child-Computer Interaction]]></source>
<year>2018</year>
<volume>18</volume>
<page-range>47-58</page-range></nlm-citation>
</ref>
<ref id="B42">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[SEITER]]></surname>
<given-names><![CDATA[L.]]></given-names>
</name>
<name>
<surname><![CDATA[FOREMAN]]></surname>
<given-names><![CDATA[B]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Modeling the learning progressions of computational thinking of primary grade students]]></article-title>
<source><![CDATA[In: ANNUAL INTERNATIONAL ACM CONFERENCE ON INTERNATIONAL COMPUTING EDUCATION RESEARCH, 9., 2013, San Diego. Proceedings [&#8230;]]]></source>
<year>2013</year>
<page-range>59-66</page-range><publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B43">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[SMITH]]></surname>
<given-names><![CDATA[L.]]></given-names>
</name>
<name>
<surname><![CDATA[CORDOVA]]></surname>
<given-names><![CDATA[J.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Weighted primary trait analysis for computer program evaluation]]></article-title>
<source><![CDATA[Journal of Computing Sciences in Colleges]]></source>
<year>2005</year>
<volume>20</volume>
<numero>6</numero>
<issue>6</issue>
<page-range>14-9</page-range><publisher-loc><![CDATA[Evansville ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B44">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[SRIKANT]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
<name>
<surname><![CDATA[AGGARWAL]]></surname>
<given-names><![CDATA[V]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[A system to grade computer programming skills using machine learning]]></article-title>
<source><![CDATA[In: ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, 20., 2014, New York. Proceedings [&#8230;]]]></source>
<year>2014</year>
<page-range>1887-96</page-range><publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B45">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[WANGENHEIM]]></surname>
<given-names><![CDATA[C. G. V.]]></given-names>
</name>
<name>
<surname><![CDATA[HAUCK]]></surname>
<given-names><![CDATA[J. C. R.]]></given-names>
</name>
<name>
<surname><![CDATA[DEMETRIO]]></surname>
<given-names><![CDATA[M. F.]]></given-names>
</name>
<name>
<surname><![CDATA[PELLE]]></surname>
<given-names><![CDATA[R.]]></given-names>
</name>
<name>
<surname><![CDATA[ALVES]]></surname>
<given-names><![CDATA[N. D. C.]]></given-names>
</name>
<name>
<surname><![CDATA[BARBOSA]]></surname>
<given-names><![CDATA[H.]]></given-names>
</name>
<name>
<surname><![CDATA[AZEVEDO]]></surname>
<given-names><![CDATA[L. F.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[CodeMaster - Automatic assessment and grading of app inventor and snap! programs]]></article-title>
<source><![CDATA[Informatics in Education]]></source>
<year>2018</year>
<volume>17</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>117-50</page-range><publisher-loc><![CDATA[Vilnius ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B46">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[WEINTROP]]></surname>
<given-names><![CDATA[D.]]></given-names>
</name>
<name>
<surname><![CDATA[WILENSKY]]></surname>
<given-names><![CDATA[U]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Using commutative assessments to compare conceptual understanding in blocks-based and text-based programs]]></article-title>
<source><![CDATA[In: THE ELEVENTH ANNUAL INTERNATIONAL CONFERENCE, 2015, Omaha. Proceedings [...]]]></source>
<year>2015</year>
<page-range>101-10</page-range><publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B47">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[WING]]></surname>
<given-names><![CDATA[J. M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Computational thinking]]></article-title>
<source><![CDATA[Communications of the ACM]]></source>
<year>2006</year>
<volume>49</volume>
<numero>3</numero>
<issue>3</issue>
<page-range>33-5</page-range><publisher-loc><![CDATA[New York ]]></publisher-loc>
</nlm-citation>
</ref>
<ref id="B48">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[ZUR-BARGURY]]></surname>
<given-names><![CDATA[I.]]></given-names>
</name>
<name>
<surname><![CDATA[PÂRV]]></surname>
<given-names><![CDATA[B.]]></given-names>
</name>
<name>
<surname><![CDATA[LANZBERG]]></surname>
<given-names><![CDATA[D]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[A nationwide exam as a tool for improving a new curriculum]]></article-title>
<source><![CDATA[In: INNOVATION AND TECHNOLOGY IN COMPUTER SCIENCE EDUCATION CONFERENCE, 18., 2013, Canterbury. Proceedings [...]]]></source>
<year>2013</year>
<page-range>267-72</page-range><publisher-loc><![CDATA[New York ]]></publisher-loc>
<publisher-name><![CDATA[ACM]]></publisher-name>
</nlm-citation>
</ref>
</ref-list>
</back>
</article>
