tangible interaction

The Language Puzzle is an interactive, multimodal language learning game for children aged approximately five to eight. Its goal is to promote language learning through a playful environment, particularly through visual, tactile, and auditory learning.

topic selection

At the beginning, we decided on major core topics to explore more deeply based on the mind map we had created in our course. The choice fell on language, identity, and understanding emotions/feelings. After another brainstorming session as a pair, we developed two project ideas.

Idea 1: The Emotion Garden for Little Big Feelings

The concept included a pegboard with plants that can be assembled individually. Children can build a small garden with different plants. The various plants differ in textures, edges, and colors — these characteristics symbolize different feelings. The plants can be freely combined, allowing individual emotional states to be represented. Integrated NFC chips detect which plant parts have been connected. Light and sound modules then respond, giving the child feedback and allowing them to empathize with how they feel or what they are currently expressing.

Idea 2: Language Puzzle

The second idea is a language learning game for children starting preschool. It involves various puzzle pieces that represent words in different languages. A picture card indicates which words are being sought. When the puzzle pieces are placed on the board, they are recognized through integrated NFC chips. Upon contact, the board plays the spoken word and displays a light signal to indicate whether the word matches the picture.

Decision for one project and initial considerations for implementation

After presenting both topics and receiving feedback, we decided to proceed with the “Language Puzzle” project. We saw great potential in this idea to teach children something meaningful and lasting.

Following this, we asked ourselves several questions about how to approach and implement the project:

• Which languages do we want to include?
• How many words do we want to implement, and which ones?
• What shape should the cards have?
• What materials will we use to build them?
• What technology do we need?

Motivation and Goal

Our goal was to develop a learning toy that works on a multimodal level and offers children diverse opportunities for interaction. Through playful experience, we want to spark joy in learning languages while simultaneously promoting language skills and vocabulary development. Inclusion was especially important to us: with our Language Puzzle, all children—regardless of their language background—should be able to learn together and have fun.

Research

We then delved deeper into research to examine what needs to be considered when developing a toy and how children best learn a new language. Here are some interesting papers on the topic: https://files.eric.ed.gov/fulltext/EJ1357958.pdf https://www.dgff.de/assets/Uploads/ausgaben-zff/ZFF-2021/Heft_ZFF_2_2021_Schluer.pdf https://www.dgff.de/assets/Uploads/ausgaben-zff/ZFF-2021/Heft_ZFF_2_2021_Jakisch_Hopp_Thoma.pdf https://tuprints.ulb.tu-darmstadt.de/21936/1/zif-3231-binanzer.pdf

From our research, we concluded—and thereby confirmed our previous approach—that it is beneficial to support children in a multimodal way. For our project, we use both visual and auditory feedback. Through additional tactile interactions, we also reinforce what has been learned.

Golden Circle - Why, How, What

To keep our project goal clear, we worked with Simon Sinek’s Golden Circle.

Why (Why are we doing this?) Children learn best when they engage all their senses. Instead of just listening or memorizing, they should see, hear, feel, and act. The goal is to make multilingual education more inclusive, intuitive, and child-friendly—for every child, regardless of their background.

How (How are we doing this?) The aforementioned multimodal elements of the project support language development. We combine picture and word cards that allow tactile interaction through a puzzle-like system. Interaction triggers direct feedback via visual and auditory signals, providing immediate responses for correct and incorrect matches.

What (What is the product?) We are developing a language-promoting wooden puzzle. By combining different picture and word cards, it provides acoustic and visual feedback to support children in learning.

Target Audience

The target group of the project is children aged approximately five to eight years, that is, children in preschool as well as first and second grade. Many children in this age range are transitioning from learning letters to beginning to read. With the support of an adult companion, preschool children can already use the game, especially for word-picture matching and listening to the spoken language output. For independent play, children should be able to recognize and read basic words in their native language.

Process

Demonstrator 1 We built the first demonstrator out of blue foam. Our main goal was to test the basic shape and structure of our idea. During this phase, several questions arose: How can the technology be installed so that lights are clearly visible and sounds are easy to hear? What shape should the final cards have?

We spent a lot of time discussing the shape of the final cards, especially in feedback sessions with other groups. The problem with typical puzzle-piece shapes was that they couldn’t be used as flexibly or inclusively. For example, if you look at the generated sketch, only the German word “Apfel” could be placed in the middle with the picture card next to it. However, it should be irrelevant where children place the picture and word cards on the board. We wanted to make the game more general with a different shape, so that other languages could be combined easily. In this demonstrator, we worked with rectangular cards. While these worked, they were quite boring for this context. Using such a general shape caused the puzzle character of the game to be lost, and errors could easily occur that weren’t related to incorrect matching. During feedback sessions, the idea of a corner-shaped card was suggested. This shape offers enough space to clearly display the terms and can only be placed in one direction, which brings back the puzzle character.

Demonstrator 2 For the second demonstrator, we aimed to test the rough game concept for the first time with children. We used wood and processed it with a laser cutter. Here, we tested the corner shape for the cards. The terms were printed on both sides so children sitting opposite each other could play together. We decided to create cards for four different languages: German, English, French, and Ukrainian. For the first testing, we prepared picture and word cards for three terms: tree, apple, and house. Questions that came up during the construction, testing, and interim presentation: Are the different languages color-coded? If yes, how will this be implemented practically? How is the LED feedback designed? Does it pulse first before indicating if the card is correct? How many terms will we develop and which ones?

Demonstrator 3 In the third demonstrator, we incorporated the light channels for the first time. To better diffuse the light and make it easier to dim, we used tracing paper. However, the light channels were still quite large and no longer created the feeling of pleasant lighting. This definitely had to be improved for the next demonstrator. This demonstrator was also made in the final size of 32 cm by 17.5 cm.

Selection of Terms For our demonstrator, we developed a set of five terms (chair, book, apple, board, bread). Each term has a picture card and four word cards. We focused on languages currently commonly spoken and relevant to the children’s environment: German, English, French, and Ukrainian. The chosen terms come from children’s everyday surroundings. They are things the children are familiar with and already use in their normal speech. This creates a direct connection between the game, language learning, and their world.

Speech Output In our first attempt, the speech output consisted of individual words generated by AI. During testing, it became clear that the playback was too short. If children were briefly distracted, they would miss what was said. Therefore, we now use full sentences for playback. These are longer, and children hear the words directly in context. The voices speaking the sentences differ by language to better distinguish the languages from each other.

Technical Setup

Components For the technical implementation of the project, the right components first had to be selected. At the heart of the system is a Raspberry Pi 4. Three RFID readers were connected to it using jumper cables. Additionally, two LED strips were used for visual feedback, and a speaker was used for auditory feedback. Power is supplied by a power bank.

Development The programming was done using the Python programming language. At the beginning of development, the focus was on implementing control of the RFID readers. A particular challenge was the fact that, by default, only two SPI interfaces were available: /dev/spidev0.0 and /dev/spidev0.1. To be able to operate three RFID readers in parallel, a third SPI interface (/dev/spidev0.2) was successfully set up by configuring and activating an additional GPIO pin accordingly. Once this technical hurdle was overcome, the logic for reading the NFC chips could be implemented. The three readers do not scan for NFC chips simultaneously; instead, they do so sequentially in very short time intervals. This means that only one reader is active at any given moment. Next, the sound output was implemented. First, the respective .mp3 files had to be saved on the Raspberry Pi and clearly assigned to the corresponding NFC chips. Playing the audio files itself then proved to be straightforward. Another aspect was controlling the LED lighting. This first required properly soldering the cables to the LED strips to ensure a stable connection to the Raspberry Pi. Afterward, it became possible to control each LED individually and precisely. However, the most effort was required for implementing the complete game logic. This involved defining exact sequences for when specific lighting effects should appear, at what moment certain sounds should be played, and how to handle various special cases. All of these aspects were taken into account and implemented during the programming phase.

Testing

Kindergarten Visit on May 19, 2025 The children participating in the first testing with our second demonstrator were between four and six years old. Some were already in preschool and will start school this year. Many of the children were very open to the game and eager to start putting some pieces together right away. They were also curious about the new languages and actively asked what was written on the cards. As expected, it was observed during testing that the game is better suited for children starting preschool rather than younger kids. Preschoolers can recognize and read the first letters, and some could even read short words with help. What still needs improvement is the presentation of the terms on the cards. Children initially learn uppercase letters only; they had trouble recognizing lowercase letters or confused letters with one another. The font used on the cards also needs to be revised because it contributed to some letter mix-ups (e.g., a capital “I” was mistaken for a lowercase “l,” or vice versa).

Play Session at the University on July 1, 2025 The children from the kindergarten came to the university for final testing. The testing of our demonstrator went mostly well. The demonstrator was sturdy enough and withstood the children’s handling, even when they were a bit rough with it. According to our observations, the multimodal feedback was very well received. The children were able to understand what the green and red lights meant and matched the corresponding sounds accordingly. After successfully pairing a match, they usually stacked the cards neatly and set them aside. For one or two children, the feedback came a bit too late—they didn’t want to wait that long and took the cards out again. However, for the majority, the buildup with light and sound worked well. The children even started repeating what they heard. The size of the toy also impressed us. The children could easily hold the cards and, thanks to the size, realized they didn’t have to be overly careful and could play freely. Overall, the testing went well, and we believe that if this toy were available, it would be well suited for daycares, schools, and homes.

Final Product

The final product consists of a wooden demonstrator with an integrated plexiglass panel. The wood provides a pleasant tactile feel, as is appreciated in toys, and the plexiglass slightly dims the built-in light, giving everything a soft, comfortable touch. For better handling, the cards were also made slightly larger. Technically, the demonstrator works very reliably. It clearly and audibly plays back all the terms, and the lights are easily visible even in daylight. Feedback is provided both visually and acoustically, engaging multiple senses. This creates an intuitive and motivating learning environment for children. Additionally, the demonstrator is built to be sturdy and withstands intensive use by children. The combination of robust materials and appealing design makes it attractive for both educational institutions and private use.

Evaluation and Outlook

The collaboration within our team went very well and smoothly. We were able to reliably hold weekly meetings to work continuously on our project. The demonstrator still offers potential for further development. Especially regarding color design, more could be achieved in a later production phase. Our idea was to distinguish the different languages through various colors to visually separate them more clearly. Content-wise, expansion is also conceivable: the sets could be supplemented with additional terms. Moreover, it would be possible to integrate more languages. Another exciting idea would be to add a microphone, allowing children to repeat the words aloud and enabling the system to recognize whether they pronounced them correctly.