Avaliku sektori innovatsioon
TalTech prioriteetne teadussuund
Klassifikaator (Frascati)
Uurimisrühma juht
Uurimisrühma liige
Doktorant
Võtmesõna
avaliku sektori innovatsioon
digitaalne transformatsioon
jätkusuutlikkuse valitsemine
avaliku-erasektori koostöö
Ülevaade
Rühma uurimisfookuses on avaliku sektori innovatsiooni mudelite ning praktiliste lahenduste edasiarendamine ja nende mõjude kriitiline analüüs. Selle kiiresti kasvava valdkonna üldine rahvusvaheline uurimisfookus on mh keskendunud küsimusele, kuidas avaliku sektori juhtimispraktikad, organisatsiooniline ülesehitus ning riigi-ühiskonna koostöövormid peaksid kohanema suurte ühiskondlike, tehnoloogiliste ja kliimakriisist tõukuvate muutuste ja nende tagajärgedega. Rühma tänased projektid ja tegevused keskenduvad nn kaksikpöörde (rohe- ja digipööre) valitsemise võimekuste, iseseisvate kogukondade võimestamise, vastutustundliku innovatsiooni ning innovatsiooni soodustavate riigihangete teemadele.
Tähtsamad tulemused
2023. aastal: • pakuti senini kõige põhjalikuma käsitluse integreeritud digiteenuste toimimise eelduseks olevatest koostööstrateegiatest ning sõnastab ja klassifitseerib Belgiat käsitleva empiirilise uurimuse tulemusena nii strateegiad kui ka nende toimimispõhimõtted.• Internetiplatvormid, mis võimaldavad mitte-elanikel ettevõtteid registreerida teise riiki, tekitavad uudseid valitsemisväljakutseid ja konflikte majanduspoliitika ning muude riiklike eesmärkide vahel. Eesti näitel demonstreeriti, millised valitsemiskonfliktid tekivad ja miks need tekivad. Uuring näitab, et selliste poliitikakonfliktide lahendamine eeldab teadlikke investeeringuid haldusvõimekustesse alates kvalifitseeritud personalist kuni andmevahetuse ja organisatsioonide vahelise koordineerimiseni.• Uuriti avaliku ja erasektori partnerluse ülesehituse mõju tehnoloogilisele innovatsioonile. Uuring arendab välja kaks vastandlikku hüpoteesi selle kohta, kuidas konkreetsete partnerluse omadused viivad tervishoiuteenuste innovatsioonini.• Põhinedes Eesti, Soome, Iirimaa ja Portugali taastava õiguse juhtumianalüüsidele, näidati, et digitaliseerimine tähendas ametnike jaoks suunavamat rolli ning kodanikele suuremat vastutust teenuse korraldamisel.
Seotud projektid
Seotud struktuuriüksus
Teadusgrupiga seotud publikatsioonid
- Lember, V., Brandsen, T., Tõnurist, P. The potential impacts of digital technologies on co-production and co-creation // Public management review (2019) vol. 21, 11, p. 1665-1686.
https://doi.org/10.1080/14719037.2019.1619807 - Randma-Liiv, T., Lember, V. (eds.). Engaging citizens in policy making : e-participation practices in Europe. Cheltenham ; Northampton : Edward Elgar Publishing, 2022. xiii, 295 p.
https://doi.org/10.4337/9781800374362 https://www.ester.ee/record=b5493342*est - Wouters, S., Janssen, M., Lember, V., Crompvoets, J. Strategies to advance the dream of integrated digital public service delivery in inter-organizational collaboration networks // Government information quarterly (2023) vol. 40, 1, art. 101779, 14 p.
https://doi.org/10.1016/j.giq.2022.101779 - Kattel, R., Lember, V., Tõnurist, P. Collaborative innovation and human-machine networks // Public management review (2020) vol. 22, 11, p. 1652-1673.
https://doi.org/10.1080/14719037.2019.1645873 - Lember, V., Kattel, R., Tõnurist, P. Technological capacity in the public sector : the case of Estonia // International review of administrative sciences (2018) vol. 84, 2, p. 214-230.
https://doi.org/10.1177/0020852317735164 - Pereira, G. V., Schuch de Azambuja, L. Smart sustainable city roadmap as a tool for addressing sustainability challenges and building governance capacity // Sustainability (2022) vol. 14, 1, art. 239.
https://doi.org/10.3390/su14010239 - Lember, V., Randma-Liiv, T., Vooglaid, K.M. Engaging citizens in policy making : the potential and challenges of e-participation // Engaging citizens in policy making : e-participation practices in Europe. Cheltenham ; Northampton, MA : Edward Elgar, 2022. p. 1-10.
https://www.ester.ee/record=b5493342*est https://doi.org/10.4337/9781800374362.00006 - Kirs, M., Lember, V., Karo, E. Technology transfer in economic periphery : emerging patterns and policy challenges // Review of policy research (2021) vol. 38, 6, p. 677−706.
https://doi.org/10.1111/ropr.12437 - Randma-Liiv, T., Lember, V. Management and organization of e-participation : synthesis from 15 European initiatives // Engaging citizens in policy making : e-participation practices in Europe. Cheltenham ; Northampton, MA : Edward Elgar, 2022. p. 260-279.
https://doi.org/10.4337/9781800374362.00025 https://www.ester.ee/record=b5493342*est - Randma-Liiv, T., Vooglaid, K.M., Lember, V. Framework for analysis of the management and organization of e-participation initiatives // Engaging citizens in policy making : e-participation practices in Europe. Cheltenham ; Northampton, MA : Edward Elgar, 2022. p. 11-24 : ill.
https://doi.org/10.4337/9781800374362.00007 https://www.ester.ee/record=b5493342*est - Aavik, G., Mayer, A., McBride, K., Krimmer, R. Is government welfare able to change? Analysing efforts to co-create an improved social welfare system through taking advantage of a collaborative economy // Proceedings of the 52st Hawaii International Conference on System Sciences : HICSS52 : January 8-11, 2019, Grand Wailea, Maui. Honolulu : University of Hawai'i at Manoa, 2019. p. 2954-2963.
https://scholarspace.manoa.hawaii.edu/handle/10125/59732 - Mikheeva, O., Tõnurist, P. Co-Creation for the reduction of uncertainty in financial governance : the case of monetary authority of Singapore // Halduskultuur: The Estonian journal of administrative culture and digital governance. [Tallinn] : Ragnar Nurkse Department of Innovation and Governance, Tallinn University of Technology, 2019. p. 60-80. (Halduskultuur : The Estonian journal of administrative culture and digital governance ; vol. 19, no. 2).
https://www.ester.ee/record=b2157203*est https://doi.org/10.32994/hk.v19i2.199 - Tõnurist, P., Valdmaa, K., Raudla, R. Impact of climate discourse on national scientific networks in energy technologies : the case of Estonian science and industry linkages // Halduskultuur : the Estonian journal of administrative culture and digital governance (2019) vol. 20, 1, p. 20-45 : ill.
https://doi.org/10.32994/hk.v20i1.203 http://halduskultuur.eu/journal/index.php/HKAC/issue/view/26/20-1-full-pdf https://www.ester.ee/record=b1447450*est - Kattel, R., Cepilovs, A., Lember, V., Tõnurist, P. Indicators for public sector innovations : theoretical frameworks and practical applications // Administrative culture = Halduskultuur = Административная культура = Verwaltungskultur = Hallintokulttuuri. Vol. 19, no. 1. [Tallinn] : Ragnar Nurkse School of Innovation and Governance, Tallinn University of Technology, 2018. p. 77-104 : tab. (Administrative culture = Halduskultuur = Административная культура = Verwaltungskultur = Hallintokulttuuri; vol. 19, no. 1).
https://doi.org/10.32994/ac.v19i1.208 https://halduskultuur.eu/journal/index.php/HKAC/article/view/208 - Schuch de Azambuja, L., Pereira, G., Krimmer, R. Clearing the existing fog over the smart sustainable city concept: highlighting the importance of governance // Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance : ICEGOV 2020, 23-25 September, 2020 : Online Conference. Danvers : ACM, 2020. p. 628–637.
https://doi.org/10.1145/3428502.3428595 - Lember, V., Petersen, O.H., Scherrer, W., Agren, R. Understanding the relationship between infrastructure public-private partnerships and innovation // Annals of public and cooperative economics (2019) vol. 90, 2, p. 371-391.
https://doi.org/10.1111/apce.12232 - McBride, K., Aavik, G., Kalvet, T., Krimmer, R. Co-creating an Open Government Data Driven Public Service : the case of Chicago's Food Inspection Forecasting Model // Proceedings of the 51st Hawaii International Conference on System Sciences : 3-6 January, 2018, Waikoloa Village, Hawaii, U.S.A.. Honolulu : University of Hawai'i at Manoa, 2018. p. 2453-2462 : ill.
http://hdl.handle.net/10125/50197 - Mergel, I., Kattel, R., Lember, V., McBride, K. Citizen-oriented digital transformation in the public sector // dg.o 18 : Proceedings of the 19th Annual International Conference on Digital Government Research : Governance in the Data Age : Delft, Netherlands, May 30-June 01, 2018. New York : ACM, 2018. art. 122, 3 p. (ACM International Conference proceedings series).
https://doi.org/10.1145/3209281.3209294 - Lember, V. The increasing role of digital technologies in co-production and co-creation // Co-production and co-creation : engaging citizens in public services. New York : Routledge, 2018. p. 115-127. (Routledge critical studies in public management).
https://www.taylorfrancis.com/books/e/9781351792578 - Kattel, R., Mazzucato, M. Mission-oriented innovation policy and dynamic capabilities in the public sector // Industrial and corporate change (2018) vol. 27, 5, p. 787–801.
https://www.ucl.ac.uk/bartlett/public-purpose/sites/public-purpose/files/iipp-wp-2018-05.pdf https://doi.org/10.1093/icc/dty032 - Mazzucato, M., Kattel, R. COVID-19 and public-sector capacity // Oxford review of economic policy (2020) vol. 36, S1, p. S256-S269.
https://doi.org/10.1093/oxrep/graa031 - Mazzucato, M., Kattel, R., Ryan-Collins, J. Challenge-driven innovation policy : towards a new policy toolkit // Journal of industry, competition and trade (2020) vol. 20, p. 421-437.
https://doi.org/10.1007/s10842-019-00329-w - Ainsalu, J., Müür, J., Soe, R.-M. et al. State of the art of automated buses // Sustainability (2018) vol. 10, 9, art. 3118, 34 p. : ill.
https://doi.org/10.3390/su10093118 - Soe, R.-M., Drechsler, W. Agile local governments : experimentation before implementation // Government information quarterly (2018) 35, 2, p. 1-13 : ill.
https://doi.org/10.1016/j.giq.2017.11.010 - Soe, R.-M., Müür, J. Mobility acceptance factors of an automated shuttle bus last-mile service // Sustainability (2020) vol. 12, 13, art. 5469, 19 p. : ill.
https://doi.org/10.3390/su12135469 - Bellone, M., Ismailogullari, A., Kantala, T., Soe, R.-M. et al. A cross-country comparison of user experience of public autonomous transport // European transport research review (2021) vol. 13, 1, art. 19.
https://doi.org/10.1186/s12544-021-00477-3 - Veskioja, K., Soe, R.-M., Kisel, E. Implications of digitalization in facilitating socio-technical energy transitions in Europe // Energy research & social science (2022) vol. 91, art. 102720.
https://doi.org/10.1016/j.erss.2022.102720 - Kattel, R., Drechsler, W., Karo, E. How to make an entrepreneurial state : why innovation needs bureaucracy. New Haven ; London : Yale University Press, 2022. xvii, 270 p.
https://www.ester.ee/record=b5500699*est https://doi.org/10.12987/9780300235371 https://doi.org/10.2307/j.ctv2t5xg83 - Drechsler, W., Kattel, R. Debate: The developed civil servant-providing agility and stability at the same time // Public money and management (2020) vol. 40, 8, p. 549-551.
https://doi.org/10.1080/09540962.2020.1729522 - Agriesti, S. A. M., Soe, R.-M., Saif, M. A. Framework for connecting the mobility challenges in low density areas to smart mobility solutions: the case study of Estonian municipalities // European transport research review (2022) vol. 14, 1, art. 32.
https://doi.org/10.1186/s12544-022-00557-y - Holloway, C., Morgado Ramirez, D. Z., Bhatnagar, T., Oldfrey, B., Morjaria, P., Moulic, S. G., Ebuenyi, I. D., Barbareschi, G., Meeks, F., Massie, J., Kattel, R., et al. A review of innovation strategies and processes to improve access to AT : looking ahead to open innovation ecosystems // Assistive technology (2021) vol. 33, S1, p. S68-S86.
https://doi.org/10.1080/10400435.2021.1970653 - Soe, R.-M. Mobility in smart cities : will automated vehicles take it over? // Smart governance for cities : perspectives and experiences. Cham : Springer, 2020. p. 189-216. (EAI/Springer Innovations in Communication and Computing).
https://doi.org/10.1007/978-3-030-22070-9_10 - Sell, R., Soe, R.-M., Wang, R., Rassõlkin, A. Autonomous vehicle shuttle in Smart City testbed // Intelligent System Solutions for Auto Mobility and Beyond : Advanced Microsystems for Automotive Applications 2020. Cham : Springer, 2021. p. 143–157. (Lecture Notes in Mobility (LNMOB)).
https://doi.org/10.1007/978-3-030-65871-7_11 - Raudla, R., Cepilovs, A., Kuokštis, V., Kattel, R. Fiscal policy learning from crisis : comparative analysis of the Baltic countries // Journal of comparative policy analysis : research and practice (2018) vol. 20, 3, p. 288–303 : tab.
https://doi.org/10.1080/13876988.2016.1244947 - Albala, S. A., Kasteng, F., Eide, A. H., Kattel, R. Scoping review of economic evaluations of assistive technology globally // Assistive technology (2021) vol. 33, S1, p. S50-S67.
https://doi.org/10.1080/10400435.2021.1960449 - Sarv, L., Kibus, K., Soe, R.-M. Smart city collaboration model : A case study of university-city collaboration // Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance : ICEGOV 2020, 23-25 September, 2020 : Online Conference. Danvers : ACM, 2020. p. 674−677.
https://doi.org/10.1145/3428502.3428601 - Sarv, L., Soe, R.-M. Transition towards Smart City : the case of Tallinn // Sustainability (2021) vol. 13, 8, art. 4143, 18 p.
https://doi.org/10.3390/su13084143 - Soe, R.-M., Sarv, L., Gasco-Hernandez, M. Systematic mapping of long-term urban challenges // Sustainability (2022) vol. 14, 2, art. 817.
https://doi.org/10.3390/su14020817 - Bellone, M., Ismailogullari, A., Müür, J., Nissin, O., Sell, R., Soe, R-M. Autonomous driving in the real-world : the weather challenge in the Sohjoa Baltic Project // Towards connected and autonomous vehicle highways : technical, security and social challenges. Cham : Springer Nature, 2021. p. 229–255. (EAI/Springer Innovations in Communication and Computing).
https://doi.org/10.1007/978-3-030-66042-0_9 - Soe, R.-M. Smart cities : from silos to cross-border approach // International journal of e-planning research (IJEPR) (2018) vol. 7, 2, p. 70-88.
https://doi.org/10.4018/IJEPR.2018040105 - Mazzucato, M., Qobo, M., Kattel, R. Building state capacities and dynamic capabilities to drive social and economic development : the case of South Africa // The future of the South African political economy post-COVID 19. : Palgrave Macmillan, 2022. p. 43-74. (International political economy series).
https://doi.org/10.1007/978-3-031-10576-0_3 - Soe, R.-M., Ruohomäki, T., Patzig, H. Urban open platform for borderless smart cities // Applied sciences (2022) vol. 12, 2, art. 700.
https://doi.org/10.3390/app12020700 - Raudla, R., Cepilovs, A., Kattel, R., Sutt, L. The European Union as a trigger of discursive change : the impact of the structural deficit rule in Estonia and Latvia // Central European journal of public policy (2018) vol. 12, 2, 15 p : ill.
https://doi.org/10.2478/cejpp-2018-0006 - Soe, R.-M. Innovation procurement as key to cross-border ITS pilots // International journal of electronic governance (2020) vol. 12, 4, p. 387-403 : ill.
https://doi.org/10.1504/IJEG.2020.10026305 https://doi.org/10.1504/IJEG.2020.112585 - Pataccini, L., Kattel, R., Raudla, R. Introduction: Europeanization and financial crisis in the Baltic Sea region: implications, perceptions, and conclusions ten years after the collapse // Journal of Baltic studies (2019) vol. 50, 4, p. 403−408.
https://doi.org/10.1080/01629778.2019.1681126 - Kattel, R., Mazzucato, M., Haverkamp, K., Ryan-Collins, J. Industriestrategie der nächsten generation für Deutschland // Wirtschaftsdienst (2020) vol. 100, p. 757-762.
https://doi.org/10.1007/s10273-020-2759-3 - Raudla, R., Mjøset, L., Kattel, R., Cepilovs, A., Mikheeva, O., Tranøy, B. S. Different faces of fiscal bureaucracy // Halduskultuur: The Estonian journal of administrative culture and digital governance. [Tallinn] : Ragnar Nurkse School of Innovation and Governance, Tallinn University of Technology, 2018. p. 5-36. (Halduskultuur: The Estonian journal of administrative culture and digital governance ; vol. 19, no. 1).
https://doi.org/10.32994/ac.v19i1.177 https://www.ester.ee/record=b2157203*est - Soe, R.-M., Schuch de Azambuja, L., Toiskallio, K., Nieminen, M., Batty, M. Institutionalising smart city research and innovation : from fuzzy definitions to real-life experiments // Urban Research & Practice (2021) vol. 15, 1, p. 112-154.
https://doi.org/10.1080/17535069.2021.1998592 - Kalda, K., Sell, R., Soe, R-M. Use case of Autonomous Vehicle shuttle and passenger acceptance analysis // Proceedings of the Estonian Academy of Sciences (2021) vol. 70, 4, p. 429-436 : ill.
https://doi.org/10.3176/proc.2021.4.09 - Karo, E., Kattel, R. Public administration, technology and innovation : government as technology maker? // Public Administration in Europe. The Contribution of EGPA. Cham : Palgrave Macmillan, 2019. p. 267-279. (Governance and public management).
https://doi.org/10.1007/978-3-319-92856-2_25 - Mazzucato, M., Kattel, R. Grand challenges, industrial policy, and public value // The Oxford handbook of industrial policy. : Oxford University Press, 2020. p. 311-336.
https://doi.org/10.1093/oxfordhb/9780198862420.013.12 - Mazzucato, M., Kattel, R., Ryan-Collins, J. Las políticas de innovación impulsadas por retos : hacia nuevas herramientas de política // Cuadernos del cendes (2021) 107, p. 25-52.
https://dialnet.unirioja.es/servlet/articulo?codigo=8196830 - Kalda, K., Sell, R., Soe, R.-M. Self-driving shuttle bus use case in city of Tallinn // IOP conference series : materials science and engineering (2021) vol. 1140, art. 012047, 6 p. : ill.
https://doi.org/10.1088/1757-899X/1140/1/012047 - Burlamaqui, L., Kattel, R. Introduction - capitalism, socialism and democracy at 75 : new interpretations and new dimensions // Schumpeter’s capitalism, socialism and democracy : a twenty-first century agenda. London ; New York : Routledge, 2019. 18 p. (Routledge studies in the history of economics).
https://doi.org/10.4324/9781315618043-1 - Vainio, T., Hodson, E., Sayún, M.N., Soe, R.-M. Decoding the smart city // Interaction design and architecture(s) (2021) issue 50, p. 59-62.
https://doi.org/10.55612/s-5002-050-002pfs - Soe, R-M. Smart governance in urban mobility process // Science and Technologies for Smart Cities : 6th EAI International Conference, SmartCity360°, Virtual Event, December 2-4, 2020 : proceedings. Cham : Springer, 2021. p. 542-552. (Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering ; 372).
https://doi.org/10.1007/978-3-030-76063-2_36 - Soe, R.-M. Challenging diffusion of digital receipts // 2021 Eighth International Conference on eDemocracy and eGovernment (ICEDEG) : Quito, Ecuador, 28-30 July 2021. Danvers : IEEE, 2021. p. 110−117.
https://doi.org/10.1109/ICEDEG52154.2021.9530887 - Kattel, R., Takala, V. The case of the UK's Government Digital Service : the professionalisation of a paradigmatic public digital agency // Digital government : research and practice (2023) vol. 4, 4, art. 28.
https://doi.org/10.1145/3630024 - Kattel, R., Burlamaqui, L. (eds.) Schumpeter’s capitalism, socialism and democracy : a twenty-first century agenda. London ; New York : Routledge, 2019. xii, 317 p. (Routledge studies in the history of economics).
https://doi.org/10.4324/9781315618043 - Sarv, L., Soe, R.-M. Piloting smart city solutions in very small, small and medium-sized municipalities. The Estonian case study // ICEGOV '22 : Proceedings of the 15th International Conference on Theory and Practice of Electronic Governance. New York : ACM, 2022. p. 475-482.
https://doi.org/10.1145/3560107.3560179 - Soe, R-M. Facilitating cross-sectoral digital product development // Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance : ICEGOV 2020, 23-25 September, 2020 : Online Conference. Danvers : ACM, 2020. p. 795-798.
https://doi.org/10.1145/3428502.3428623 - Surva, L. Maintaining the ideals of co-production during rapid digitalisation : a comparative case study of digital restorative services in Estonia, Finland, Ireland and Portugal // Voluntas (2023) Vol. 34, Issue 4, p. 693 - 707.
https://doi.org/10.1007/s11266-022-00502-6 - Soe, R-M. Replication and upscaling of Smart Cities in Academia and practice: concepts, barriers and enablers // 24th Annual International Conference on Digital Government Research: dg.o 2023 : Together in the unstable world: Digital government and solidarity, Gdańsk, Poland, 11-14 July, 2023 : proceedings. New York : ACM, 2023. p. 115−121.
https://doi.org/10.1145/3598469.3598481 - Andreoni, A., Kattel, R. States of innovation : how the state shapes production transformation // Handbook of industrial development. : Edward Elgar Publishing, 2023. p. 382-402.
https://doi.org/10.4337/9781800379091.00033 - Baudens, P., Masso, A., Soe, R-M. Women’s (im)mobility strategies and digital platform adoption: the case study of employees doing desk work in Pune, India // Gender, technology and development (2023) vol. 27, 3, p. 423-443.
https://doi.org/10.1080/09718524.2023.2260651 - Fernandez-Monge, F., Barns, S., Kattel, R., Bria, F. Reclaiming data for improved city governance : Barcelona’s new data deal // Urban Studies (2023)
https://doi.org/10.1177/00420980231204835 - Schuch de Azambuja, L., Soe, R-M. Devising an urban learning centre for municipalities in Eastern Partnership countries // New Sustainable Horizons in Artificial Intelligence and Digital Solutions : 22nd IFIP WG 6.11 Conference on e-Business, e-Services and e-Society, I3E 2023, Curitiba, Brazil, November 9–11, 2023, Proceedings. Cham : Springer, 2023. p. 403-417.
https://doi.org/10.1007/978-3-031-50040-4_30 - Lember, V., Crompvoets, J. Public services and management in the digital age // Public management and governance. : Routledge, 2023. p. 141-152.
- Drechsler, W., Kattel, R., Chafik, S. Islamic public administration and Islamic public value : towards a research agenda // Public policy and administration (2023)
https://doi.org/10.1177/09520767231201799 - Lember, V. Public-private partnerships for service provision // Elgar Encyclopedia of Services. Cheltenham : Edward Elgar, 2023. p. 173–176.
https://www.e-elgar.com/shop/gbp/elgar-encyclopedia-of-services-9781802202588.html https://doi.org/10.4337/9781802202595 - Fröhlich, K. A., Soe, R-M., Maoneke, P. B., Jain, K., Pinomaa, A., Nieminen, M. Extending the “Smart City” concept to small-to-medium sized Estonian municipalities: initiatives and challenges faced // Information Systems : 19th European, Mediterranean, and Middle Eastern Conference, EMCIS 2022, Virtual Event, December 21–22, 2022 : proceedings. Cham : Springer, 2023. p. 264–276. (Lecture notes in business information processing ; 464).
https://doi.org/10.1007/978-3-031-30694-5_20 - Ylönen, M., Drechsler, W., Lember, V. Online incorporation platforms in Estonia and beyond : how administrative spillover effects hamper international taxation // Transnational corporations (2023) vol. 30, 1, p. 79-101.
https://doi.org/10.18356/2076099x-30-1-3