MENU
  • School
  • Lifestyle
  • Question
  • Safari
  • Entertainment
  • Sports
  • Curling
  • Contact
Language
  • English
  • 日本語
Takegawa Lab. - xFuture Lab.
Takegawa Lab. - xFuture Lab.
  • School
  • Lifestyle
  • Question
  • Safari
  • Entertainment
  • Sports
  • Curling
  • Contact
  • English
  • 日本語
  2. Entertainment of the Future

Entertainment of the Future

Greetings

Currently entertainment is undergoing a dramatic transformation.
Generative AI enables us to single-handedly both produce and enjoy artwork, stories, music, and more.
This in an age in which anyone can produce a highly polished creative work, and
in professional spheres works are being produced that are on another level altogether.
Not only that, technology such as AI and XR is facilitating the proposal of new entertainment experiences every day.
You could say this is entertainment’s ‘Cambrian explosion’

However, the popularity of entertainment is not decided by quality and novelty alone.
The most impactful musical experience may take the form of an old man’s inexpert playing,
the whole-hearted performance of child, or a song produced by someone who has overcome great adversity, because it is hearing not only the sound, but also the story (process) of the person producing it that moves our hearts.
High quality, new experiences, personal stories… this is the evolution of the state of entertainment.
Entertainment of the Future, while proposing new forms of entertainment, also seeks and shares the true forms of amusement and passion, through questions, technologies and forms of expression.
Why not join us in searching for the essence of entertainment?
This work is partly proceeding as collaborative research with the creative collective MPLUSPLUS.

Prof. Yoshinari Takegawa

Project

(Click thumbnails for details)
If you are using an ad blocker, the content may not be displayed properlyｘ

Painted Musical Instrument
Create any electronic musical instrument by drawing
PerformMask
A versatile expression-changing mask for use in performance
Dance Instrument and Projection Dance
Dance Instrument and Projection Dance
Baatarina
Support for finding your position in stage performance
Projection Mapping LED
The possibility to produce light displays on even bumpy surfaces
Wearable Piano Illuminated Piano
New form of expression and experience for pianists
Airstic Drum
A hybrid drum combining the real and the virtual
Unit Instruments
Like LEGO with instruments
OneParka
Enabling you to share the excitement of a live music performance from the comfort of your own home
BrainOneParka
A Parka that combines brainwaves and vibration to share everyone’s excitement
Mobile Clavier
Enabling you to play varied harmonies with just a small number of keys
Pokerepo GO
A portable TV station
AIdoridesu7 (AI Switcher)
“Switching” manipulating multiple cameras

© Future University Hakodate, Takegawa Laboratory, All Rights Reserved.

Pers-lens



Understanding perspective changes the way you draw backgrounds

Learning ‘perspective’, which is vital for drawing backgrounds, proves a significant hurdle for beginners. Even if a learner can understand the concepts of perspective, such as eye level (the height of one’s line of sight) and perspective lines (lines that converge towards the distance), working out how to apply these to drawing actual scenery is more problematic. This is why we developed ‘Pers-lens’, an AR glasses-type system that uses AR technology to overlay and display eye level and perspective lines on an actual view. When you look at a building or interior space while wearing Pers-lens, you can confirm your eye level, which represents the height of your eyeline, and see perspective lines that converge towards the distance. This provides a visual comprehension of perspective in your daily environment, giving you an objective understanding of the relationship between your own viewpoint and the makeup of your surrounding environment. Furthermore, by making it easier to obtain a sense of perspective, this system increases fundamental background-drawing skills.

パースレンズ



遠近がわかると、背景の描き方が変わる。

背景イラスト制作に不可欠な「遠近法」の習得は、初学者にとって大きな壁となっています。アイレベル（視線の高さ）やパース線（奥行き方向へ収束する線）といった遠近法の概念は、知識として理解できても、現実の風景にどのように当てはめればよいのかわからないという課題があります。そこで、AR技術を用いて現実の風景にアイレベルとパース線を重ねて表示するARグラス型システム「パースレンズ」を開発しました。「パースレンズ」を使用すると、建物や室内空間を見た際に、目線の高さを示すアイレベルや、奥行き方向に収束するパース線をリアルタイムで確認できます。これにより、普段の生活空間の中で遠近法を視覚的に捉えることが可能となり、自身の視点と空間構造の関係を客観的に理解できるようになります。さらに、遠近法を感覚として身につけやすくなることで、背景描画の基礎力を上げることができます。

Painted Musical Instrument



Create any electronic musical instrument by drawing 

Painted Musical Instrument is a device that enables you to create any kind of electronic instrument you like, simply by drawing on paper. Usually, we use instruments that already ordinarily exist in the world, but if we were able simply to create original instruments that do not yet exist anywhere in the world, our dreams would expand. For example, you could easily make an instrument suited to your own body size, which might allow you to play difficult parts of music with ease, or play with a greater range of expression. Besides this, when you apply the Painted Musical Instrument device, being able to turn a staircase, chair, doorknob, handrail, or any household item, into an instrument need not be a mere dream. 50 or 100 years in the future, the instruments most widely used may not be those designed by well-informed instrument experts, but those casually designed by children or people with no musical experience.

Accomplishment
Takegawa, Y., Fukushi, K., Machover, T., Terada, T., and Tsukamoto, M., “A Prototyping Support System for the Creation of Painted Musical Instruments,” International Journal of Arts and Technology, Vol. 8, No. 4, pp. 382–402 (Dec. 2015).
Takegawa, Y., Fukushi, K., Machover, T., Terada, T., and Tsukamoto, M., “Construction of a Prototyping Support System for Painted Musical Instruments,” Proceeding of ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE2012), pp. 384–397 (Nov. 2012).

PerformMask


A versatile expression-changing mask for use in performance

Masks are often used in performances such as children’s plays. As a mask has a fixed appearance, the expressions it can convey are limited. Perform Mask can flexibly change its appearance, expressing for example changes of expression, age, gender, meaning that one mask can be used for multiple scenes or roles. First of all, ink that changes color by heat transmisison is applied to the part of the mask that one wants to change. Then, copper tape is simply adhered to the underside of the area to which ink was applied. Thus, anybody can create an original ‘mask of a hundred faces’.

Accomplishment
Masui, M., Takegawa, Y., and Hirata, K., “Dynamic Appearance Augmentation Method that Enables Easy Prototyping of Masks for Performance,” Proceedings of the Augmented Humans 2022 (AHs2022), pp. 267–275 (March 2022).

Dance Instrument and Projection Dance



Dance Instrument and Projection Dance

Dance Instrument is a new expression tool for dancers. In breakdancing, dancers take part in dance battles in which opponents compete by demonstrating their dance skills. Dance Instrument plays impressive sound effects at appropriate moments during dance, in time with the BGM. Projection Dance produces slick visual effects. Producing sound effects and visual effects just at the right moment – there is actually an amazing technology behind that.

Accomplishment
Fujimoto, M., Fujita, N., Takegawa Y., Terada, T., and Tsukamoto, M., “Musical B-boying: a Wearable Musical Instrument by Dancing,” Proceeding of the 7th International Conference on Entertainment Computing (ICEC2008), pp. 155–160 (Sep. 2008).
Fujimoto, M., Fujita, N., Takegawa, Y., Terada, T., and Tsukamoto M., “A Motion Recognition Method for a Wearable Dancing Musical Instrument,” Proceeding of IEEE International Symposium on Wearable Computers (ISWC2009), pp. 11–18 (Sep. 2009).

Baatarina



Support for finding your position in stage performance

A dancer or singer’s stage performance involves confirming his/her position on the stage in advance, during rehearsal. Performances will not necessarily always be given on the same stage. Stages come in all shapes and sizes and this greatly affects a performer’s position on the stage, especially in the case of dancers or singers. There are times when sufficient rehearsal time cannot be spent on confirming position. To enable a performer to check whether he/she is in the correct position, we attach an HMD (a glasses-type display). We support performance by displaying information on the HMD that allows the performer to confirm where he/she is on the stage and whether he/she is in the proper position.

Accomplishment
Ikeda, J., Takegawa, Y., Terada, T., and Tsukamoto, M., “Evaluation on Performer Support Methods for Interactive Performances Using Projector,” iiWAS/MoMM2009 Special issue in Journal of Mobile Multimedia (JMM), Vol. 6, No. 3, pp. 207–226 (Sep. 2010).

Projection Mapping LED



The possibility to produce light displays on even bumpy surfaces

It is very difficult to project onto, for example, a Christmas tree, which is has an uneven surface and is not white. There are LED cables that can be wrapped around a Christmas tree, aren’t there? With these, it is easy to make all the lights shine or flash, but, when it comes to creating a pattern, things get a lot more complicated. Projection Mapping LEDs are hi-tech LEDs with sensors that detect the color of light from a projector. By lighting up only the LEDs that the light from a certain specialized projector has shone on, you can make a pattern appear.

Accomplishment
Nakata, M., Kodama, K., Fujita, N., Takegawa, Y., Terada T., Tsukamoto, M., Hosomi, S., and Nishio, S., “Design and lmplementation of a Ubiquitous Optical Device Controlled with a Projector,” Proceeding of the 6th International Conference on Advances in Mobile Computing and Multimedia (MoMM2008), pp. 130–135 (Nov. 2008).

Wearable Piano and Illuminated



New form of expression and experience for pianists

Wearable Piano is not just a piano that has been turned into a dress. The keyboard really produces sound. With Illuminated Piano, the dress lights up attractively in time to the playing of Wearable Piano. A future where you can color your daily life with music? It could be just around the corner!

Accomplishment
Takegawa, Y., Tsukamoto, M., Oe, M., and Nishio, S., “Wearable Piano: Realizing Wearable Fashion Collaborating with Piano,” Proceeding of IEEE International Symposium on Wearable Computers (ISWC 2005), Demonstration (Business Innovation Center Osaka, Osaka, Oct. 2005).

Airstic Drum



A hybrid drum combining the real and the virtual

When it comes to travelling to perform in various places, transporting a large and heavy drum kit is problematic. Using an electric drum, comprising sensors attached to drumsticks so that the performer can play as on an `air drum’, allows a performer to go out and perform anywhere. However, as drumming involves many techniques that can only be performed using the actual surface of a drum, it cannot be said that the electric drum allows the user freely to perform any piece of music. This is where Airtstic Drum comes in. Airstic Drum provides a surface to act as the parts of the drum kit that are often used, while those parts less frequently used are recreated with the electric drumsticks. This is a hybrid of the real and the virtual that makes it possible to take the drum anywhere and deliver high quality performance.

Accomplishment
Kanke, H., Takegawa, Y., T., Terada, T., and Tsukamoto, M., “Airstic Drum: A Drumstick for Integration of Real and Virtual Drums,” Proceeding of ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE2012), pp. 57–69 (Nov. 2012).

Unit Instruments



Like LEGO with instruments

When playing acoustic instruments, music is produced by altering the pitch, strength, length or timbre of sounds through different playing techniques. In contrast, electronic instruments imitate the playing techniques of acoustic instruments then use sensors or buttons to input pitch, strength, length or timbre of notes into a computer which interprets these signals and outputs sound. In Unit Instruments, the results of disassembling the input parts of various electronic instruments into individual components are called `units’. By altering the combination of these units, electronic instruments affording new playing techniques can be created, such as an instrument that is a combination of a guitar and a piano, or a piano with additional keyboards.

Accomplishment
Takegawa, Y., Terada, T., and Tsukamoto, T., “UnitKeyboard: An Easily Configurable Compact Clavier,” Proceeding of International Conference on New Interfaces for Musical Expression (NIME2008), pp. 289–292, (June 2008).
Maruyama, Y., Takegawa, Y., Terada, T., and Tsukamoto, M., “UnitInstrument: Easy Configurable Musical Instruments,” Proceeding of International Conference on New Interfaces for Musical Expression (NIME2010), pp. 7–12 (June 2010).

OneParka



Enabling you to share the excitement of a live music performance from the comfort of your own home

ONEParka is a Parka that increases the sense of unity between participants in online live music performances. In an online performance, participants do not feel the sense of unity that should be experienced at a live performance, because they are unable to share the mood and excitement that are unique to an actual live event. ONEParka solves this problem. This device consists of a Parka with built-in miniature vibration transmission speakers that produce vibration along with sound, which vibrate in time to the live music. In addition, the device measures excitement levels based on the wearer’s pulse rate, expression, perspiration, and movement, and alters the intensity of the vibration in proportion to the excitement level. By using ONEParka, online live performance participants can share each other’s excitement and experience the sense of unity that could previously only be obtained at a real-world performance.

Accomplishment
Anbo, Y., Imai, R., Matsui, R., Yanagisawa, Y., Takegawa, Y., and Hirata, K., “A Clothing-Type Wearable Device that Promotes a Sense of Unity for Viewers of Online Live Performances,” In Proceedings of the 25th HCI International Conference (HCII ’23), pp. 209-–224 (July, 2023).

BrainOneParka



A Parka that combines brainwaves and vibration to share everyone’s excitement

Brain One Parka is a next-generation parka that increases the sense of unity in online and VR concerts. Utilising the technology of the preceding ONE Parka, we also install brainwave sensors to analyse users’ brainwaves in real time. The system estimates the level of excitement from the analysis results and shares the heightened emotion of other viewers through vibrations. This makes it possible to experience a sense of presence, as though in a real concert venue, while feeling the hype of the concert in one’s body. We introduced real-time feedback by brainwave analysis, which was not implemented in ONE Parka, and further increased the sense of unity between viewers. With Brain One Parka, you can get a sense of enjoying a concert together with everyone else, no matter where you are. Let’s use Brain One Parka to enjoy concerts that connect us to others, even when we’re alone.

Mobile Clavier



Enabling you to play varied harmonies with just a small number of keys

Many people who play musical instruments want to be able to perform for lots of people, whenever, wherever. However, ordinary keyboard instruments are large and heavy, making them difficult to carry around. While there are smaller keyboard instruments, the range of sounds that can be played is limited, making it difficult to achieve a satisfying performance. This is where Mobile Clavier, which is portable and enables you to play any piece of music, comes in handy. It is a small keyboard instrument, comprising just 15 white keys and 12 black keys, yet, by operating the foot pedal to alter the range as you play, it is possible to perform pieces covering three octaves, such as a Turkish March. In addition, to prevent any confusion for the player, the ranges played by the left hand and right hand, respectively, are distinguished by the way the LEDs light up. Looking foward, we think that incorporating ever-involving AI technology into Mobile Clavier will remove the need for the foot pedal. With Mobile Clavier, the performers of the future will be able to deliver satisfying performances to even more people, in even more places.

Accomplishment

Takegawa, Y., Terada, T., Tsukamoto, M., and Nishio, S., “Mobile Clavier: New Music Keyboard for Flexible Key Transpose,” Proceeding of International Conference on New Interfaces for Musical Expression (NIME2007), pp. 82–87 (June 2007).

Pokerepo GO



A portable TV station

Creating footage for TV programs etc. requires a lot of people and time. There are many necessary processes, beginning with an editors’meeting to decide what kind of footage to take, what to film and how to present it; then research and preparation, filming, editing, and, in the case of any narration or voiceover, studio recording. Furthermore, filming involves various equipment and many people, such as someone to direct, someone to operate the camera and someone to record sound. Pokerepo GO is a tool to enable this kind of video to be created by one person. Filming and editing can be performed simultaneously with a single Pokerepo GO unit.

Accomplishment
Matsumura, K. and Takegawa, Y., “Reporting Solo: A Design of Supporting System for Solo Live Reporting,” Proceeding of ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE2016), pp. 1–7 (Nov. 2016).
Takegawa, Y., Matsumura, K., and Manabe, H. “PokeRepo Go++: One-man Live Reporting System with a Commentator Function,’’ Proceedings of the 2019 ACM International Conference on Interactive Experiences for TV and Online Video (TVX ’19). Association for Computing Machinery, New York, NY, USA, pp. 230-–238 (June 2019).

AIdoridesu7 (AI Switcher)



“Switching” manipulating multiple cameras

In a live television broadcast, footage is taken with various cameras and the image aired is simultaneously decided by choosing the optimum camera image from among these. The task of choosing between these camera shots is called `switching’. Switching is work carried out by professional `switchers’, based on their experience and know-how, and is very difficult for amateurs. However, it is thanks to this process that even live video is easy for viewers to comprehend. AIdoridesu7 is the name of a switching support system. Essentially it is a system to assist amateurs with the difficult task of switching. The results of a survey showed that experienced switchers take occurrences (events) in video as hints for selecting and switching between camera images. Therefore, AIdoridesu7 provides a function for automatic detection of events and an event-based interface that is simple to operate even for beginners, enabling any person to broadcast optimum images easily.

Accomplishment
Endo, S., Takegawa, Y., Funaki, A., Matsumura, K., Hirata, K., and Igarashi, T., “Construction of a Switching Support System for Live Broadcast of Oral Presentation,” Journal of Information Processing, Vol. 62, No. 3, pp. 1–9 (Mar. 2021).