MENU
  • オープンラボ
  • School
  • Lifestyle
  • Question
  • Safari
  • Entertainment
  • Sports
  • Talent
  • Contact
Language
  • English
  • 日本語
Takegawa Lab. - xFuture Lab.
Takegawa Lab. - xFuture Lab.
  • オープンラボ
  • School
  • Lifestyle
  • Question
  • Safari
  • Entertainment
  • Sports
  • Talent
  • Contact
  • English
  • 日本語
  2. School of the Future

School of the Future

Greetings

Our lives have been drastically changed, and will certainly continue to be changed, by technologies such as Artificial Intelligence, VR (Virtual Reality), cryptocurrency, IoT, automatic driving, and so on. While the introduction of these technologies has given rise to new social problems, such as privacy and security issues, there are also many examples of improvements brought about by technology, for example increased self-expression by new SNSs, an unprecedented connection between people by new communication tools, and increased assurance and safety by driving assistance.

For we who are living in an age in which lifestyle, common knowledge and customs are being transformed more and more by technology, it is necessary to learn and apply new things each day.

School of the Future is not only for a specified age group, such as elementary school children. People out in the working world, leading busy lives, retired people spending their old age restfully, even new-born babies, all surely want to learn new skills. It is simply that the skills a person wants to learn have become varied, encompassing many fields from human relations, to fashion, to games.

For all people who feel they want to extend their own abilities, our hope is that School of the Future will tangibly contribute to enabling even just one more experience of the joy of ‘I did it!’.

Prof. Yoshinari Takegawa

Project

(Click thumbnails for details)
If you are using an ad blocker, the content may not be displayed properly

Learning Tool Design

Design of tools to enable learners to learn efficiently

Projection Piano
The shortcut to piano mastery
Score Gazing
The eyes don’t lie
Me, the score, and the keyboard
Piano playing analyzed from gaze information
Rendan
Joining the dots to connect piano lessons and practice
Erasure-oriented Illustration Method
Slow and steady wins the race when it comes to skill acquirement
Afterward Auxiliary Lines
Lines that follow your drawing
AnoPers
Imagining and drawing a motif seen from a different perspective
Table Cutting Board
With this, you too can handle a knife like a top chef
Tablet Handwriting
Transcription in which a learner practices using a brush on a tablet
Reflection Stamps
A more enjoyable way to reflect on your drawings. A simple and positive feedback tool.
Maai

Teaching Method Design

Design of teaching methods for applications, systems and games that support learning

Teacher who is good at prediction
How much do I need to practise to improve?
Teacher with a knack for analogy
Teaching through comparison to daily movements
Teacher who deliberately overlooks, or even covers up, your mistakes
To keep learners motivated
Teacher who doesn’t overlook the little things
Advice from a viewpoint you cannot see with your own eyes
Multimodal teacher
Teaching through a variety of methods
Teacher who lies, out of love
For graduation from the support system (teacher)
Teacher who reminds you of physical limitations
Enabling a teacher to provide singing instruction appropriate to their student’s lung capacity
Aesthetic AI
An AI judgement system imbued with an aesthetic sense, to visualize divergence in artistic expression
Just practicing as much as you can or thinking before you start practicing – which is more efficient?
Otoato

© Future University Hakodate, Takegawa Laboratory, All Rights Reserved.

Maai


Me, the score, and the keyboard



Piano playing analyzed from gaze information

In piano performance, where the player is looking is important. That’s why, in this research, we developed a system that uses eye-tracking technology to analyze eye movement during performance and enables the player to reflect objectively on their own playing. This system combines the image from a camera attached to the top of the keyboard with the gaze data obtained by two eye trackers, and displays this as four images on a computer screen. The top right shows the player’s face, the bottom right shows the player’s hands, the top left shows where the player is looking on the musical score, and the bottom left shows where the player is looking on the keyboard. This arrangement makes it clear when, for example, the player’s gaze is jumping from right to left while playing, or shifting between the score and the keyboard again and again. Furthermore, in places that the player is replaying over and over, the points on the score that they are focusing on, or a place where they are stuck on the left-hand part, can be understood from gaze information. In this way, by looking at their playing and the movement of their own gaze, simultaneously, a player can objectively analyze the habits and problems of their performance, which helps them to practice efficiently.

Tablet Stone



Realizing a curling stone that maintains a stable information display no matter how much the stone is spinning
Tablet Stone is an application that was developed for players of curling. This application begins with the placing of a tablet on top of a curling stone. On the tablet, information such as the stone’s movement speed, rotation speed and number of rotations is displayed. This information is important for, for example, the sweepers (the players who sweep the ice with brushes). However, because a curling stone spins as it travels, it is difficult for the sweepers to read information displayed on top of the stone. By making the information display rotate in response to the rotation speed of the stone as it slides across the ice, we make it easy for players to read the data. But seeing is believing, so check out the demo video!

Accomplishment
Guanyu, C., Mori, H., Takegawa, Y., Aihara, S. and Masui, F. (2024). Design and Implementation of a Stone Rotation Measurement System with IMU Sensor and Stone Behavior Presentation System. In Proceedings of the 12th International Conference on Sport Sciences Research and Technology Support, pp. 140–147 (Nov. 2024).

Aesthetic AI



An AI judgement system imbued with an aesthetic sense, to visualize divergence in artistic expression

Aesthetic AI is a system that compares a person’s musical performance with a model performance and visualizes any divergence in musical expression. Previous AI excelled at generating images and video, but had difficulty with accurately evaluating forms of human expression. Our system analyses the waveform data of a musical performance and estimates the extent to which it differs from the model (ideal) performance, then presents the difference visually, with images. The basis of this technology is the anomaly detection technology used in the manufacturing industry. We apply technology that is used, for example, to detect a tiny wrinkle in a piece of fabric or a miniscule dent in a pill, and use it to discern minute differences in musical performance. The AI picks up minor divergences, that ordinary people would not notice, and makes an objective judgement. This AI with a sense of aesthetic appreciation has the potential, in future, to prove useful in not just music, but a wide range of creative fields, such as art and design.

Accomplishment
Matsui, R., Yanagisawa, Y., Takegawa, Y., “Detecting Performance Anomalies: A Novel Method for Analyzing Musical Differences in WAV and MIDI Data,” Journal of Information Processing, Vol. 66, No. 5, pp. 1–11 (May. 2025).

Reflection Stamps



A more enjoyable way to reflect on your drawings. A simple and positive feedback tool.
Reflection Stamps is a tool that makes it easy to reflect on an illustration you have drawn. People who don’t like reflecting on their work often feel that ‘I only notice the bad parts’ or ‘I don’t want to look at my drawings.’ To enable such people to enjoy reflection, in Reflection Stamps you can place text stamps on the parts of an illustration that you like or want to improve. For example, by looking at your illustration and adding stamps such as “The hair looks soft; it looks good!” or “I want to adjust the balance of the eyes a little,” it’s easy to sort out the good parts and the parts you want to alter. As the result of an experiment, we found that using Reflection Stamps makes reflection take less time-consuming and encourages a positive feeling during reflection. In the future, we plan to increase the variety of stamps to enable even more diverse expression, and to improve the usability of the tool. Using Reflection Stamps allows anybody to reflect positively on their own illustrations.

Anomaly Detection


Accomplishment
Matsui, R., Yanagisawa, Y., Takegawa, Y., “Detecting Performance Anomalies: A Novel Method for Analyzing Musical Differences in WAV and MIDI Data,” Journal of Information Processing, Vol. 66, No. 5, pp. 1–11 (May. 2025).

Just practicing as much as you can or thinking before you start practicing – which is more efficient?



Otoato
I believe most people aiming to become good at the piano practice by simply playing over and over again. However, our investigation showed that, even if it means cutting down on the time spent actual playing, what’s important is to think about how you should play in order to improve, while practicing. That’s why we created ‘otoato’, a system to support practice. Otoato sorts out the content of your practice to make it easy to understand, just like a teacher. For example, in the ‘heat map’ function, places where you made a lot of mistakes during practice are colored red, while places where there were few mistakes are colored blue. By looking at the heat map, you can tell at a glance which parts you have difficulty with and where you make a lot of mistakes. Furthermore, there is a function that enables you to add memos about things you have noticed or need to think about. Utilizing memos such as “Next I should practice this part more” allows you to think of personalized practice methods. Using otoato enables you to reflect on your practice while improving more efficiently at the same time. Why not have fun rethinking your piano practice with otoato?

Keyboard Cover



Turns even the quietest person into a chatterbox
A computer keyboard is a multi-functional device that can be used not just for typing, but also for operations involved in gaming, music, design, and so on. However, most applications largely involve operations using specific keys or mouse buttons, which are different for each application, meaning that operation is counter-intuitive and takes time to get used to.
This is where Keyboard Cover comes in handy. Keyboard Cover comprises a plastic sheet with small extrusions, which is fitted over a computer keyboard. Attaching Keyboard Cover enables you to add new interfaces, such as dials or sliders, to the ordinary keys, transforming a single keyboard into various types of controllers. Keyboard Cover can reduce instances of accidentally pressing the wrong key and make operation more intuitive, thus improving operational efficiency. Also, as the layout of the cover can be customized to suit the purpose of an application, you can create the optimum operation environment for you. If you’re struggling with keyboard operation, why not let Keyboard Cover inhabit your keyboard?

Afterward Auxiliary Lines



Lines that follow your drawing

Achieving correct balance is one important aspect of drawing. Accordingly, many people draw guide lines in advance, as a means of drawing with correct balance. However, the problem with conventional auxiliary lines is that it becomes necessary to adjust the lines again and again as the drawing progresses. This is where ‘Afterward Auxiliary Lines’ comes in. In Afterward Auxiliary Lines, auxiliary lines are displayed automatically in response to the illustration that is being drawn, removing the need to alter the auxiliary lines manually. Furthermore, drawing that is completely dependent on auxiliary lines does not exercise the drawer’s powers of observation, but with Afterward Auxiliary Lines it is possible to draw and make alterations while looking at one’s own illustration, the object or motif that is being observed and the auxiliary lines, enabling the user to draw while learning observation skills. Why don’t you try this easy way of improving your drawing skills with the assistance of these clever auxiliary lines?

Projection Piano



The shortcut to piano mastery

To become able to play the piano, there are many hurdles to overcome, such as learning to read music, practicing fingering and maintaining rhythm, and learning how to apply strength when pressing the keys. Furthermore, it is very hard to learn these skills alone, causing some players to become disheartened partway through. Projection Piano is the answer to such a problem. Projection Piano is designed to reduce the hurdles that tend to trip up people starting to learn the piano and make piano practice fun. By practicing with Projection Piano, a player should be able to enjoy mastering the piano without becoming disheartened along the way.

Accomplishment
Takegawa, Y., Terada, T., and Tsukamoto, M., “Design and Implementation of a Piano Practice Support System using a Real-Time Fingering Recognition Technique,” Proceeding of International Computer Music Conference (ICMC2011), pp. 1–8 (Aug. 2011).

Score Gazing



The eyes don’t lie

You may not find it difficult, when sat next to someone who is playing the piano, to follow his place on the score with your eyes. However, when a computer system tries to do the same thing it is not so straightforward. Playing tempo changes constantly according to expression, and there are times when a note may not be played or a mistake made. In practise the player may also miss out difficult parts and repeatedly practice the same places. By using not only sound and input to the musical instrument, but also combining these with gaze information, the score gazing system follows the score as though it is right beside the player, to support performance.

Accomplishment
Terasaki, S., Takegawa, Y., and Hirata, K., “Proposal of Score-Following Reflecting Gaze Information on Cost of DP matching,” Proceeding of International Computer Music Conference (ICMC2017), pp. 144–149 (Oct. 2017).
Noto, K., Takegawa, Y., and Hirata, K., “Adaptive Score-Following System by Integrating Gaze Information,” Proceedings of 16th Sound and Music Computing Conference (SMC2019), pp. 401–406 (May 2019).

Rendan





Joining the dots to connect piano lessons and practice

When taking piano lessons, it is essential to practice individually at home. However, there is a problem, in that it is difficult for the teacher accurately to grasp how the student has practiced at home. This is where Rendan comes in. Rendan is an application that visualizes the state of a student’s piano practice so that it can be understood at a glance. On the home screen, a calendar is displayed, and on each date there is a donut graph expressing amount of practice and number of keying errors. Detailed figures can be checked by selecting a date. In addition, it is possible to confirm not only the amount of practice on a single day, but also to compare that day with the amount of practice conducted to date, which makes it easy to understand how much a student has progressed. Furthermore, amount of practice can even be confirmed at the level of bars or notations, with places that have been practiced a lot displayed in a darker color, enabling the teacher to confirm amount of practice at a glance. While Rendan is aimed at piano teachers, it can also be used for a parent to check their child’s practice, or to help students themselves to get an accurate understanding of the state of their own practice.

Accomplishment
Mori, H., Sasaki, M., Noto, K., Takegawa, Y. and Hirata, K. “Design and Implementation of a Practice Record Visualization System Using Piano Performance Tracking Technology,” In Proceedings of the 8th International Conference on Computer-Human Interaction Research and Applications, pp. 1–15 (Nov. 2024).
Sasaki, M., Noto, K., Takegawa, Y., and Hirata, K., “Practice Progress Visualization System to Support Piano Teachers,” In Proceedings of International Conference on Music Perception and Cognition (ICMPC ’23), pp. 267 (Aug. 2023).

Erasure-Oriented Illustration Method



Slow and steady wins the race when it comes to skill acquirement

There is a saying (slow and steady wins the race) which means that, if you want to hurry, you can sometimes get there more quickly in the end by taking the long way round. What happens if we apply the essence of this to skill acquisition (Illustration)? As an example, we considered an erasure-oriented illustration method, which incorporates the idea of ‘erasing’ as you draw. In actual practice this means, when drawing a shape like a UFO, for example, simply drawing two overlapping circles then erasing the unneeded parts. How does that sound? Don’t you think you could draw better that way than by trying to draw the shape of a UFO as it is? When you master this illustration method, you become able to draw more complex illustrations, skillfully. By taking the long way round, you are able to draw more easily and simply. Sounds like a pretty good method, don’t you think?

Accomplishment
Kurotaki, R., Takegawa, Y., and Hirata, K., “Proposal of an Erasure-Oriented Drawing Style to Develop the Ability to Copy Images,” Proceeding of International Conference on Entertainment Computing (ICEC2017), pp. 209–216 (Sept. 2017).

AnoPers



Imagining and drawing a motif seen from a different perspective

When beginners learn digital illustration, they tend to practice by copying or sketching flat images, whereas in reality the quickest way to improve is to practice drawing three dimensional motifs as much as possible. In the proposed system, the learner repeatedly performs the training task of imagining how a three-dimensional motif, displayed on a computer, looks from a specified viewpoint and drawing it. The learner can use the computer interface to change the viewpoint. However, to prevent the learner from seeing the correct answer, there is a fixed area, centered on the specified viewpoint, that cannot be viewed. This idea was partly informed by a type of spatial awareness training called mental rotation. The results of an experiment showed that the system improved users’ ability to draw angle, size and position both accurately and three-dimensionally. Having a grasp on three-dimensional drawing enables an illustrator to draw the kind of difficult compositions, involving low angles and overhead views, that are used in action-packed manga.

Table Cutting Board


 


With this, you too can handle a knife like a top chef

Tablet cutting-board is a system to improve the user’s technique of cutting ingredients with a knife. Before cutting ingredients, the thinness of slices is set in advance. When ingredients are cut on top of the tablet, the system displays whether or not they have been cut to the appropriate width, in real time on the tablet screen. Tablet cutting-board can judge the width of cut ingredients with an accuracy degree of within 0.5mm. It would not be exaggerating to say that this makes its judgement accuracy higher than that of a professional chef. Beginners often make the mistake of cutting to an appropriate width, but in a diagonal direction. Usually, it is difficult to notice when the knife is at a diagonal, but Tablet cutting-board uses sound to notify the user of whether or not the knife is horizontal. In this way, this system supports the first step towards mastering knife-handling technique. Incidentally, you may wonder whether it’s really okay to use a tablet as a cutting board, but relax; a tablet’s glass screen is a lot stronger than you’d think!

Accomplishment
Masuda, H., Kato, K., Ikematsu, K., Takegawa, Y., and Hirata, K., “Tablet Cutting Board: Tablet-based Knife-control Support System for Cookery Beginners.” In Proceedings of the Augmented Humans International Conference 2023 (AHs ’23). Association for Computing Machinery, New York, NY, USA, pp. 287–293 (March, 2023).

Tablet Handwriting



Transcription in which a learner practices using a brush on a tablet

Instances of writing by hand are decreasing, but people still enjoy receiving handwritten letters and greetings cards. To write characters neatly, it is essential to consider the correct order of penstrokes and the balance of each character. Practising writing with a brush is worthwhile, even if just to learn the history of the characters. However, in addition to the difficulty of using a brush, which is probably unfamiliar, one must consider balance and brush control while comparing with a sample image. This makes it very difficult to tell, just by looking, whether or not the character is drawn correctly. In tablet calligraphy, the user can learn character balance by placing a piece of Japanese calligraphy paper over a tablet displaying an example character, and tracing with a brush dipped in water. Characters written are automatically assigned a score, enabling users to understand their progress in terms of a numerical value.

Accomplishment
Tanaka, S., Takegawa, Y., and Hirata, K., “Proposal of a Support System for Learning Brushstrokes in Transcription for Beginners,” Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI2020), pp. 352–356 (Mar. 2020).
Nonami, J., and Takegawa, Y., “Construction of a Support System for Learning Character Balance in Transcription for Beginners,” Proceeding of IEEE Global Conference on Consumer Electronics (GCCE2014), pp. 26–30 (Oct. 2014).

Teacher who is Good at Prediction



How much do I need to practice to improve?

How much do I need to practice to improve?
Practise or study with no end in sight is hard and frustrating. Doubtless everyone has experienced a period during which their skills do not improve even though they practice, or of feeling uncertain during a slump in progress. Simply by being shown current practise outcomes, test results etc., `Teacher who is good at prediction’ estimates how a person will progress in future, from a vast amount of past records. The future prospect of improving if you practice this much, study this much, becomes an incentive to overcome the difficult periods.

Accomplishment
Yoshino, K., Takegawa, Y., Hirata, K., and Tominaga, A., “Construction of a Model for Predicting Student’s Performance in Programming Exercise Lecture,” Compute Software (JSSST), Vol. 37, No. 3, pp. 67-78 (May 2020).

Teacher with a Knack for Analogy



Teaching through comparison to daily movements

In sport or playing instruments, when using the body in a different way from in everyday life, it can be difficult to visualize what you need to do to master certain skills. `Teacher with a knack for analogy’, uses comparison to everyday actions to teach those ways of using the body that are difficult to visualize. By inputting all kinds of everyday action, this system can tell you the daily action in which the way you use your body is closest to the action of a professional. When you learn which of your everyday actions is similar to the action of a pro, you can more rapidly become competent at a sport or an instrument.

Teacher who Deliberately Overlooks, or even Covers up, Your Mistakes



To keep learners motivated

Acquirement of skills that use the body takes a long time. Noticing mistakes is the first step to achieving correct movement. A ‘computer teacher’ can unfailingly notice even slight mistakes made by learners. However, if all these mistakes are pointed out to learners, they will feel they are always being criticized and come to loathe practice. ‘Teacher who deliberately overlooks mistakes’ selects the mistakes that will be pointed out to learners, based on their proficiency levels. Furthermore, the system covers up certain mistakes, for example, by outputting the correct sound from a piano even when an incorrect key has been pressed. But you worry about whether you can actually improve by practicing in this way, don’t you? Don’t worry. You will improve.

Accomplishment
Fukuya, Y., Takegawa, Y., and Yanagi, H., “A Piano Learning Support System Considering Motivation,” Proceeding of International Computer Music Conference (ICMC2013), pp. 62–68 (Aug. 2013).

Teacher who Doesn’t Overlook the Little Things



Advice from a viewpoint you cannot see with your own eyes

When you go to have piano lessons, the teacher has to consider not only score reading and practice methods, but also fingering, posture and even your musical way of thinking. However, there is a limit to the area that a single teacher can look at. In particular, paying attention to every minute movement and placement of the fingers is very difficult. `Teacher who doesn’t overlook the little things’ scrutinizes your piano practice from viewpoints from which the teacher cannot, and gives advice. Of course, this concept is not limited to learning the piano. The system could watch over practice of other instruments and sports from every angle, to support progression.

Accomplishment
Hasegawa, A., Takegawa, Y., and Hirata, K., “Design and Implementation of a Support Tool to Find Bad Fingering Habits for Piano Teachers,” International Conference on Music Perception and Cognition (ICMPC15), pp. 1–6 (July 2018).

Multimodal Teacher



Teaching through a variety of methods

When learning skills that use the body, such as sport, playing a musical instrument, or cookery, in books there is no choice but to learn from visual expression such as text and diagrams. If you attend lectures or classes you can learn from a teacher, but with `Multimodal Teacher’ you can learn skills through experiences, more richly, thanks to a teaching presentation style produced by the combination of several components, such as sight, sound and touch.

Accomplishment
Tanaka, S., Takegawa, Y., and Hirata, K., “Proposal of a Support System for Learning Brushstrokes in Transcription for Beginners,” Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI2020), pp. 352–356 (Mar. 2020).

Teacher who Lies, out of Love



For graduation from the support system (teacher)

There are many systems to aid with the acquisition of skills in sport, music and so on, but at some point you have to become independent of the system and able to perform skills by yourself. ‘Teacher who lies, out of love’ helps you to graduate from this kind of support system (the teacher). The majority of support systems (teachers) teach the correct way of doing something, but this system begins to tell lies once the user has learned to perform a skill as taught. Being lied to causes the user gradually to come to mistrust the system (teacher). Finally the user is no longer misled by the system’s (teacher’s) lies, and can become independent.

Accomplishment
Kumaki, M., Takegawa, Y., and Hirata, K., “Design and Implementation of a Positioning Learning Support System for Violin Beginners, Using True, Vague and False Information,” Journal of Information Processing, Vol. 26, pp. 285–293 (Mar. 2018).
Kumaki, M., Takegawa, Y., and Hirata, K. “A Positioning Learning Support System using False Information and Vague Information for Violin Beginner,” Proceedings of World Conference on Educational Media and Technology (EdMedia2016), pp. 1560–1571 (June 2016).

Teacher who reminds you of physical limitations




Enabling a teacher to provide singing instruction appropriate to their student’s lung capacity

To improve singing technique and performance, it is essential to manipulate your body appropriately, such as by controlling your breathing. However, in actual singing lessons, teachers often use abstract phrases to instruct. Breathing with the physical ability of a teacher (adult) and breathing with the physical ability of a student (child) are different, thus, a student requires instruction that is appropriate to their physical ability. Therefore, we developed a band-type device for singing teachers. By singing with the band fitted around the lower part of their chest, a teacher can experience the difference between their own physical ability and that of their student. The band senses chest movements and recognizes whether the wearer is breathing in or out. Then, the system displays a graph depicting residual respiratory volume (how much breath is left). The line on the graph goes up when the user breathes in, and down when the user breathes out. For example, if the residual respiratory volume gauge is set altered from 100% to 75%, in accordance with the student’s respiratory volume, by singing while keeping their breathing within the range of the residual respiratory volume gauge, the teacher can understand how it feels for the student. In this way, the teacher becomes able to instruct in a way that is suited to their student.