Artificial Intelligence on legal language processing: using Deep Learning to find the regulatory law framework for the Third Sector

Authors

Keywords:

third sector, regulation, deep learning, natural language processing, law

Abstract

This paper deals with the application of artificial intelligence algorithms in processing legal language to identify a complete set of rules applicable to a given legal theme. In this study, we sought to delimit the regulatory framework that involves the Third Sector, based on the data set on the Brazilian regulation flow (RegBR). From the bibliographic research, machine learning techniques were applied to automate the classification of each sentence within the analyzed normative acts, allowing us to identify to what extent a norm applies to the selected topic. The BERT model with fine-tuning by a Brazilian legal dataset was highly effective, reaching 94% of precision (F1-Score and AUC). The results include a total found of 2,359 rules spread in 611 normative acts on the 1,330,190 sentences distributed in 51 thousand regulations contained in the dataset, demonstrating how the applied techniques can contribute to the improvement of the themes involved.

Downloads

Download data is not yet available.

Author Biographies

Mauricio Barros de Jesus, Accounts Court of Goiás (TCE/GO)

Master in Applied Computing with an emphasis on Data Science, University of Brasília - UnB  and a Computer Engineer, Federal University of Goiás - UFG.

André da Silva Goes, Accounts Court of Goiás (TCE/GO)

Student of the Professional Master's Program in Public Administration of the PROFIAP/UFG network (2022), a specialist in Auditing and Government Management from PUC-GO, Bachelor's degree in Computer Science, PUC-GO.

Leonardo de Guimarães Santiago, Accounts Court of Goiás (TCE/GO)

Student of the Professional Master's program in Public Administration (PROFIAP/UFG), Bachelor of Administration (PUC-GO) and specialist in Public Sector Economics and Public Sector Auditing (WPós).

Marcelo Augusto Pedreira Xavier, Accounts Court of Goiás (TCE/GO)

Specialist in External Control and Public Governance (IDP/2016) and student of the Master's Program in Human Rights at the Federal University of Goiás (PPGIDH/UFG).

Sólon Bevilacqua, Federal Univesity of Goiás (UFG)

Professor and Coordinator of the Professional Master's Degree in Public Administration - PROFIAP/UFG, graduated in Administration (UFRGS), Specialist in Production Engineering (UFRGS), Master in Administration (UFU) and Doctor in Psychology from PUC-GO.

References

ABRUCIO, F. L. (2006). Contratualização e organizações sociais: reflexões teóricas e lições da experiência internacional. Debates GV Saúde, 1(10), 24–27.

Advocacia-Geral da União (2018). Manual de Boas Práticas Regulatórias. Brasília: AGU. Retrieved from https://drive.google.com/file/d/1lKjBXT7WQpXWAdM_3xZClsD-BnqvU8-X/view

AMARAL, F. (2016). Aprenda Mineração de Dados: teoria e prática. Alta Books: Rio de Janeiro.

ANSELL, C., & TORFING, J. (2016). Introduction: theories of governance. In: Ansell C., Torfing, J. (editors). Handbook on Theories of Governance. Elgar Online. https://doi.org/10.4337/9781782548508.00008

ARANHA, M. I. (2014). Manual de Direito regulatório: fundamentos de Direito regulatório (2nd ed., Coleford, Gloucestershire, United Kingdom: Laccademia Publishing Limited.

ARANHA, M. I. (2019). Teoria jurídica da regulação: entre escolha pública e captura. RDP - Revista Direito Público - Edição Especial: 15 anos. 16 (2019), 11–37.

BARBOSA, N. B., LIMA, H. de, TRINDADE, A. M. V. da, DIAS, N. V., CASTRO, M. V. de, & SANTOS, C. K. dos. (2015). As Organizações Sociais de Saúde como forma de gestão público-privada em Goiás – o caso HUANA. Revista Do Serviço Público, 66 (1), 121 - 144. https://doi.org/10.21874/rsp.v66i1.686

BERMUDES, W. L., SANTANA, B. T., BRAGA, J. H. O., & SOUZA, P. H. (2016). Tipos de escalas utilizadas em pesquisas e suas aplicações. Revista Vértices, 18 (2), 7-20.

BRASIL (1995). Plano Diretor da Reforma do Aparelho do Estado Cadernos Mare. Brasília: Presidência da República 1995. Retrieved from http://www.biblioteca.presidencia.gov.br/publicacoes-oficiais/catalogo/fhc/plano-diretor-da-reforma-do-aparelho-do-estado-1995.pdf

CHALKIDIS, I., FERGADIOTIS, M., MALAKASIOTIS, P., ALETRAS, N., & ANDROUTSOPOULOS, I. (2020). LEGAL-BERT: The Muppets straight out of Law School. Findings of the Association for Computational Linguistics: EMNLP. https://doi.org/10.48550/arXiv.2010.02559

CRACK, A. M. (2018). The Regulation of International NGOS: Assessing the Effectiveness of the INGO Accountability Charter. Voluntas 29, 419–429. https://doi.org/10.1007/s11266-017-9866-9

DEVLIN, J., CHANG, M., LEE, K., & TOUTANOVA, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.48550/arXiv.1810.04805

DIAS, G., FARIAS, J., COURA, K., DEMO, G., & ATHAYDE, A. (2022). Reflexões sobre a falta de água em algumas regiões brasileiras e a coprodução na gestão pública:: rumo a uma gestão hídrica mais efetiva. Revista do Serviço Público, 73 (1), 32 - 54. Recuperado de https://revista.enap.gov.br/index.php/RSP/article/view/4769

ELLIG, J. (2018). Why and How Independent Agencies Should Conduct Regulatory Impact Analysis. Cornell Journal of Law and Public Policy, 28 (1), 1–34. http://doi.org/10.2139/ssrn.3169413

Escola Nacional de Administração Pública (2022). RegBR. Dados Atualizados em Maio/2022 com foco em normativos regulatórios. Brasília: Enap. Retrieved from https://infogov.enap.gov.br/regbr

FERRI, C., HERNÁNDEZ-ORALLO, J., & MODROIU, R. (2009). An experimental comparison of performance measures for classification. Pattern Recognition Letters, 30 (1), 27-38. https://doi.org/10.1016/j.patrec.2008.08.010

HAHN, R. W. & TETLOCK, P. C. (2008). Has Economic Analysis Improved Regulatory Decisions? Journal of Economic Perspectives, 22 (1), 67–84. https://doi.org/10.1257/jep.22.1.67

HASTIE, T., TIBSHIRANI, R., FRIEDMAN, J., & FRANKLIN, J. (2005). The elements of statistical learning: data mining, inference and prediction. The Mathematical Intelligencer, 27 (2), 83–85. https://doi.org/10.1007/BF02985802

KO, Y., & SEO, J. (2009). Text classification from unlabeled documents with bootstrapping and feature projection techniques. Information Processing & Management, 45 (1), 70-83. https://doi.org/10.1016/j.ipm.2008.07.004

KOOP, C., & LODGE, M. (2017). What is regulation? An interdisciplinary concept analysis. Regulation & Governance, 11 (1), 95–108. https://doi.org/10.1111/rego.12094

LEVI-FAUR, D. (2011). Regulation and Regulatory Governance. In Handbook on the Politics of Regulation. [s.l.] Edward Elgar Publishing, 2011, 3–21. https://doi.org/10.4337/9780857936110.00010

LIU, Y., ZHOU, Y., WEN, S., & TANG, C. (2014). A Strategy on Selecting Performance Metrics for Classifier Evaluation. International Journal of Mobile Computing and Multimedia Communications (IJMCMC), 6(4), 20-35. http://doi.org/10.4018/IJMCMC.2014100102

LUGER, G. F (2013). Inteligência Artificial (6th ed, Daniel Vieira, Trans.) São Paulo: Pearson Education do Brasil.

MCCORMICK, C., & RYAN, N. (2019). BERT Fine-Tuning Tutorial with PyTorch. Retrieved from https://mccormickml.com/2019/07/22/BERT-fine-tuning/

MCMULLIN, C. (2021). Challenging the necessity of New Public Governance: Co-production by third sector organizations under different models of public management. Public Administration, 99 (1), 5–22. https://doi.org/10.1111/padm.12672

MINAEE, S., KALCHBRENNER, N., CAMBRIA, E., NIKZAD, N., CHENAGHLU, M., & GAO, J. (2020). Deep Learning Based Text Classification: A Comprehensive Review. https://doi.org/10.48550/arXiv.2004.03705

MONTANO, M. R., & PIRES, D. D. O. O (2019). Marco Regulatório Do Terceiro Setor: A Oferta Da Educação Infantil No Município De Porto Alegre. Práxis Educacional, 15 (31), 16-17. https://doi.org/10.22481/praxis.v15i31.4656

PHILLIPS, S., & SMITH, S. (2014) A Dawn of Convergence?: Third sector policy regimes in the “Anglo-Saxon” cluster. Public Management Review, 16 (8), 1141–1163. https://doi.org/10.1080/14719037.2014.965272

PHILLIPS, S., & HEBB, T. (2010). Financing the third sector: Introduction. Policy and Society, 29 (3), 181–187. https://doi.org/10.1016/j.polsoc.2010.07.001

SAAB, F., & SILVA, S. A. M. (2021). Evaluating the quality of Regulatory Impact Analysis: a literature review. Revista do Serviço Público, 72(b), 34 - 57. Retrieved from https://revista.enap.gov.br/index.php/RSP/article/view/5875

SANO, H., & ABRUCIO, F. L. (2008). Promessas e resultados da Nova Gestão Pública no Brasil: o caso das organizações sociais de saúde em São Paulo. Revista de Administração de Empresas, 48 (3), 64–80. https://doi.org/10.1590/S0034-75902008000300007

SHAFFER, R., & MAYHEW, S. (2019). Legal Linking: Citation Resolution and Suggestion in Constitutional Law. In Proceedings of the Natural Legal Language Processing Workshop 2019, 39–44. Minneapolis, Minnesota: Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-2205

SHEARER, C. (2000). The CRISP-DM model: the new blueprint for data mining. Journal of data warehousing, 5 (4), 13-22.

SILVA, T. T. S. E., NEVES, C. B. D., JESUS, M. B. D., & MODANEZ, W. (2022). Inteligência artificial e Jurisprudência: delimitação jurisprudencial nas decisões do TCU do conceito aberto de cláusula restritiva ao caráter. Editora Dialética.

SOH, J., LIM, H., & CHAI, I. (2019). Legal Area Classification: A Comparative Study of Text Classifiers on Singapore Supreme Court Judgments. In Proceedings of the Natural Legal Language Processing Workshop 2019, 67–77. Minneapolis, Minnesota. Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-2208

TALESH, S. (2022). Public law and regulatory theory. In Handbook on Theories of Governance. [s.l.] Edward Elgar Publishing. 104–117. https://doi.org/10.4337/9781800371972.00016

VASWANI, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L. Gomez, A., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, 6000–6010. https://doi.org/10.48550/arXiv.1706.03762

VIEIRA, J. P. T. O. (2016). Fragilidades Dos Mecanismos De Controle No Município Do Rio De Janeiro. Rio de Janeiro. FGV EBAPE - Dissertações, Mestrado Profissional em Administração Pública [366]. Retrieved from http://hdl.handle.net/10438/16060

ZHENG, L., GUHA, N., ANDERSON, B., HENDERSON, P., & HO, D. (2021). When does pretraining help? assessing self-supervised learning for law and the CaseHOLD dataset of 53,000+ legal holdings. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law (ICAIL '21). Association for Computing Machinery, New York, NY, USA, 159–168. https://doi.org/10.1145/3462757.3466088

Downloads

Published

2023-08-07

How to Cite

Barros de Jesus, M., da Silva Goes, A., de Guimarães Santiago, L., Pedreira Xavier, M. A., & Bevilacqua, S. (2023). Artificial Intelligence on legal language processing: using Deep Learning to find the regulatory law framework for the Third Sector. Revista Do Serviço Público, 74(2), 439-461. Retrieved from https://revista.enap.gov.br/index.php/RSP/article/view/8091

Issue

Vol. 74 No. 2 (2023)

Section

Artigos

License

Copyright (c) 2023 Revista do Serviço Público

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

- A RSP adota a licença Creative Commons (CC) do tipo Atribuição – Uso Não-Comercial (BY-NC).

- A licença permite que outros remixem, adaptem e criem obra licenciada, sendo proibido o uso com fins comerciais.

- As novas obras devem fazer referência ao autor nos créditos e não podem ser usadas com fins comerciais, porém não precisam ser licenciadas sob os mesmos termos dessa licença.

- Ao publicar o artigo na RSP, o autor cede e transfere para a ENAP os direitos autorais patrimoniais referentes ao artigo.

- O artigo publicado na RSP não poderá ser divulgado em outro meio sem a devida referência à publicação de origem.

- O autor que tiver o artigo publicado na RSP deverá assinar o Termo de Concessão de Direitos Autorais (em momento oportuno a editoria da Revista entrará em contato com o autor para assinatura do Termo).