We spoke with the creators of ALIA, the 100% Spanish AI, to understand its future

This Monday it was announced release of ALIA language models. The initiative has been in development for years and it is now that the first fruits are beginning to be seen, still modest, but promising. To learn more details about ALIA, at Xataka we have spoken with Martha Villegas (@MartaVillegasM), head of the Language Technologies Unit of the Barcelona Supercomputing Center (BSC). This has allowed us to clarify the status of the project, its objectives and its next challenges. To compete with ChatGPT, nothing The first thing we wanted to know is how ALIA had been created, and here Marta Villegas clarified that the model is based on the Llama architecture – Meta’s Open Source model –, “but the model has been trained from scratch and with zero initial weights“. This is important because ALIA is not a Llama-based model that has undergone a refinement or “fine-tuning” process. In those cases, this expert explained, “you start from a model trained with other data and with initialized weights, and you do it to adapt that model to your needs, either because you have more data and you want it to be better or because perhaps you want to adapt it to a particular domain. But here, he told us, “the vocabulary (set of tokens) is completely different.” In other models the corpus or training data set may be mostly in English, which causes the set of admissible tokens to be calculated through English. That, Villegas indicates, would make it adapt less efficiently to other languages. That is precisely what has been sought with ALIA: reduce the relevance of English to increase the number of 35 languages ​​of the European Union and, especially, Spanish, Catalan, Basque and Galician. How ALIA has been trained The ALIA training process began with some experiments in April 2024. It is necessary because as Villegas explained, “training is not pressing the button after feeding the data and that’s it.” It had to be taken into account that MareNostrum 5the supercomputer located at and managed by the BSC, had just come into operation at full power and there was high demand to use it. MareNostrum 5 In this training process, the ALIA project has had gradual availability of the computing capacity of MareNostrum 5. Although for a short period of time they had access to 512 of the 1,120 specialized nodes of the supercomputer, 256 nodes were used for many months and since September They are using 128 nodes, “which is a lot,” Villegas highlights. During the training process, he told us, there are so-called “checkpoints”, in which it is possible to evaluate how the training process is going. These “pauses” also allow certain training data to be updated, as in fact happened in that process in which at a given moment they introduced a new corpus with high quality that allowed them to replace some data they had. This is just the beginning: it’s time to “instruct” and “align” ALIA Villegas explained to us that ALIA is a foundational model: it is not prepared to be an alternative to ChatGPT. The latter is based on GPT-4, a much more ambitious foundational model that involved much more investment. Here we must differentiate the foundational model from the “educated” and “aligned” models with which we usually interact. As this expert told us, “ALIA-40b is a foundational model that is not instructed or aligned. For a model to be a ChatGPT and understand the conversation and have a certain memory and be “politically correct,” the foundational model (which only learns to say the next token) is “instructed” by passing a bunch of texts.” Even so, the goal is to gradually consider these options. “In March, the instructed version of ALIA-40b is expected to be launched, with a first set of open instructions,” Villegas told us. These instructions are going to be subcontracted – the ones that allow these models to be instructed – and a million euros are going to be invested in that set of instructions from scratch. This data will also be published so that it is available to institutions and developers: if it has been paid with public money, explains Villegas, it is logical that this data will also be public, something that does not usually happen with other AI models from private companies. While training AI models provides guidance on how to respond and defines the context and purpose of those responses, alignment solves problems such as avoid discriminatory biasprevent misinformation or protect privacy. Precisely this lack of alignment means that using these models in this initial phase can produce responses with errors and biases that are precisely mitigated to a great extent with this alignment phase. ALIA and the competition: it is neither a rival of ChatGPT nor does it intend to be In fact, Villegas highlights, “the objective is not to compete with ChatGPT, for that we would need 5 billion dollars.” ALIA-40b “is a good model, and a chatbot can be made in the future because the intention is to instruct and align it, but that will take time.” Within the ALIA family we have the Salamandra models (2b and 7b), smaller and more modest but which already have first instructed versions. Its performance and capacity still have room for improvement, but they are good starting points for the future. It was inevitable to ask how ALIA then intends to compete with other models, both closed and developed by private companies and Open Source models. For her “There is a demand for intermediate models that each person can then adapt to their specific use case, not everyone can use ChatGPT for reasons such as privacy or use case.” Villegas also wanted to highlight how these smaller models can have exceptional performance in specific tasks, and can work at levels of security and not sharing important data. The objective is not to compete with ChatGPT, for that we would need 5,000 million dollars Not only that, he reveals: “we also took out the … Read more

Why what we understand as “normal” development in children could be wrong

Image source, Getty Images photo caption, Due to the immense variety of components that affect the growth of a human being, it is very difficult to define something as “normal.” Item information Author, Samuel Forbes and Prerna Aneja Author’s title, The Conversation* January 14, 2025 For parents, caregivers and teachers, it is often tempting to base our understanding of a child’s development on what we believe is “normal.” We often do it without thinking, when we describe a child as “doing well” in one subject and “falling behind” in another. Whenever we make this kind of comparison, we have some kind of mental reference point in our heads: for example, a toddler should be able to climb furniture at age 2. Increasingly, child development researchers argue that the same is true in their field, the study of how behaviors and skills such as language develop. Many of the studies that claim to investigate child development, whether implicitly or explicitly, claim that their findings are universal. There may be many reasons for this. Sometimes there is a temptation to exaggerate conclusions, sometimes it can be the way readers or the media interpret the findings. The result is that what has been found in a group of children is then taken as the standard, the criterion against which future research is compared. Academic biases Image source, Getty Images photo caption, Most of the existing academic research on child development comes from Western countries. Most research on child development comes from wealthier Western countries, particularly the United States, the United Kingdom, the Netherlands, Germany, and France. Chances are, if you’ve heard of childhood development milestones, they occurred in one of these countries. This is because it can be difficult to conduct basic research on child development in developing countries, as colleagues and reviewers will ask or demand comparisons with Western populations to put findings from these regions in context. Of course, without realizing it, these colleagues and reviewers have established Western children as the norm. Complex environments Image source, Getty Images photo caption, Most academic studies on child development have been carried out in developed countries, and do not take into account development in other cultures. But is it fair to make these comparisons? One of the complicated aspects of child development research is that it occurs in a cultural and social context from which it cannot be separated. But this context is often confusing. Differences in physical environment, parenting styles, location, climate, etc. interact to shape children’s growth. In addition to these differences, there are also individual variations. These can be, for example, curiosity, shyness and neurodiversity, which can frame the way a child shapes their own learning environment. Take the field of childhood motor development: the study of how children learn to move. Many parents in particular may be familiar with charts showing when they can expect their child to sit, crawl, stand and run. The existence of these graphs makes it seem quite universal, and a child’s motor development is often judged in this way. This makes sense. Early research was concerned with finding out what was normal, and it makes sense to try to support children who might be at risk of falling behind. The time and order investigated then gave rise to the norms and scales that we still use today. Image source, Getty Images photo caption, Studies have standardized the stages of development, without taking into account that the environment affects each individual differently. Is something like the timing of motor development universal? It’s easy to imagine it could be. When there are no physical or cognitive barriers, we all learn to sit and stand, so at first glance it seems fair to say that it could be. But it turns out that the context in which children develop plays a very important role even in something as seemingly universal as this. In countries and cultures where babies routinely receive firm massages from their caregivers, such as in Jamaica, motor development accelerates. It is clear that a norm developed in one culture may not translate well to another. Beyond the rules Image source, Getty Images photo caption, Many times, research has no way to incorporate key information such as the social and cultural context of the children it studies. Clearly, the problems highlighted above are not unique to motor development. In areas such as language development or social development, the cultural component is even more pressing. There is simply no way to understand these elements of child development without also understanding the context in which they take place. Each child develops within a context and, no matter how normal our own culture may seem to us, There is no objective, context-independent standard with which we can compare other children.. That is, we should accept the disorder. If we think of normal child development as something that just happens, researchers miss understanding the dynamics of development itself. But worse, educators and caregivers may not realize that development is something we can act on, and they miss the opportunity to create change. Image source, Getty Images photo caption, Each child develops uniquely, and it is through that understanding that better results are achieved. An important part of viewing child development as intertwined with culture is that it not only means collecting data from other cultures, but involving local communities and research perspectives. Understanding communities means listening to them, empowering them and giving them space to have a voice. Moving beyond a Western-centric understanding of child development will not only benefit researchers and lead to more accurate science, but will hopefully benefit everyone who works with children around the world. *This article was published on The Conversation and reproduced here under the creative commons license. Beam click here to read the original version. Samuel Forbes is Associate Professor of Psychology at the University of Durham and Prerna Aneja is a Lecturer in Psychology at the University of East Anglia. Subscribe here to our new newsletter to receive a selection … Read more

Log In

Forgot password?

Forgot password?

Enter your account data and we will send you a link to reset your password.

Your password reset link appears to be invalid or expired.

Log in

Privacy Policy

Add to Collection

No Collections

Here you'll find all collections you've created before.