Being able to communicate with others is an essential aspect of life. Just think how frustrating it would be if you were unable to communicate with your friends or co-workers. 

An App Changed Shion’s Life

Some fantastic examples of what can be done with Book Creator by teachers or students in the special education classroom.

Special education books

In the summer of 2016, Mr. Uchida gave a presentation at the Apple Distinguished Educators Global Institute in Berlin, in which he shared how he and his fellow teachers were using the iPad in their classroom. When speaking with Professor Jim Moore from the University of Georgia in the United States one evening, he explained his findings about the decrease in Shion&#8217;s heart rate.

Being able to communicate with others is an essential aspect of life. Just think how frustrating it would be if you were unable to communicate with your friends or co-workers. Now, put yourself in the position of a child who cannot communicate her needs to her family or teachers. How frustrated would you be?

Teachers commonly use the app to communicate with Shion and her facial expressions now display more positive emotions. She has started to smile more often, and has missed fewer days of school than before the app was available.

Building upon the skills he acquired during his university training, Mr. Reisigl worked diligently to develop an app based on Mr. Uchida&#8217;s idea. Mr. Uchida was astonished, as the app addressed each of the parts of his sketch. Although there were several components involved in the process, Mr. Reisigl&#8217;s initial version of the app made it more convenient for Mr. Uchida to monitor Shion&#8217;s emotions. Photo taken by Luke D'Agostino

Professor Moore was intrigued with what Mr. Uchida was learning, but wanted to see if the process could be improved. During one of their email conversations later that year, Mr. Uchida included a sketch of his ideal app. With this in hand, Professor Moore approached a faculty member in his university&#8217;s New Media Institute who put him in touch with a student, Mr. Joe Reisigl, to see if he could create a prototype. 

Occasionally, teachers learn more from their students than they impart, and this was one of those special times. Not only Shion make us better educators, but she also reminded us of the immense power of collaboration. 

Using the app, teachers in the classroom found that Shion's heart rate decreased when her name was called - evidence that she was interested and focusing. The teachers were so excited to see this happen that they repeated the process. Shion soon started to recognize the process and began raising her arm to show that she was understanding. As expected, her heart rate increased as she raised her arm.

Shion&#8217;s teachers were always wondering whether she liked hearing tambourines or drums, or seeing images of bears or dogs. Knowing her interests would allow them to improve her educational experience. 

While Mr. Uchida would like for this app to be useful for the other students with severe disabilities, the current version needs to be improved further. One of the biggest challenges is that the app is difficult for some people, including Shion&#8217;s family members, to use, as it requires many complex technical skills. As Mr. Reisigl is no longer available to work on this project, he has recently shared all of his files. Mr. Uchida is hoping to make the app more user friendly so that it can be used to communicate with people around the world, regardless of their disabilities. 

Fortunately, there is a public school in the Saitama Prefecture in Japan for students with severe special needs, particularly students who struggle to communicate with others. The teachers in this school work together to create an environment that supports and includes all students, including those with disabilities and certain illnesses. The teachers develop lessons that incorporate drawing using technology or dancing in wheelchairs. This makes all of the students feel safe and special. As a result, they love school. 

All children at Special Education Schools are unique. As a result, their teachers must plan different goals for each one of them. This book portrays how technology solved the issues the teachers had when they taught a particular student. I would like you to see how the app helped Shion grow. I would like to thank everyone who was involved with this project. I hope this will help many children with disabilities and their supporters.

I was amazed by Takahiro&#8217;s presentation in Berlin, which was delivered in flawless English. When I tried to congratulate him that evening, I was even more amazed to learn that he had memorized his 3-minute speech in a language he didn&#8217;t know. This inspired me to become involved, and I was very pleased when Joe Reisigl accepted the challenge of turning Takahiro&#8217;s vision into reality. 

Wow. I am beyond humbled to know this app changed the way Mr. Takahiro Uchida interacted with Shion and, more importantly, has improved Shion&#8217;s life. It is incredible to know that writing a few lines of code helped change the life of someone on the other side of the globe. This truly shows how technology is improving our well-being and continues to improve our world every day.

I am extremely honored that I was invited to join this team and watch this beautiful project unfold. The joy these educators take in their work and their pure focus on helping students with severe special needs inspired me to share this story to multiple people around the world. It has been a real pleasure working with them and I look forward to seeing how this project evolves in the near future. Last but not least, this project would not have started without the guidance and encouragement of Jim. I am so grateful to have such an amazing role model. His modesty and humility inspires us all.

Shion is a student with a severe disability who occasionally was seen smiling spontaneously during class. Because Shion has difficulties speaking, the teachers could not figure out why she was smiling. Neither her parents nor her doctors could explain Shion&#8217;s random episodes of happiness. As a result, her teachers were determined to communicate with her. 

When the teacher repeated Shion&#8217;s favorite word &#8216;Ahe ahe ahe&#8230;&#8221; (roughly &#8220;ha ha&#8221; in Japanese), Shion&#8217;s heart rate decreased and she remained focused for a long time.

As Mr. Uchida was performing this initial study, Apple released their new Apple Watch, which could be used to monitor heart rate. Mr. Uchida had Shion wear it and repeated the process. Shion&#8217;s heart rate changed depending on the activity, the involvement of her teachers, and her level of interest.

After trying different ways to understand Shion better, Shion&#8217;s teacher, Mr. Takahiro Uchida, found a paper summarizing a research study by Mr. Kitajima from Chiba University. A key finding in that study was a decrease in heart rate when people responded positively to stress and were interested in something. In contrast, the heart rate increases when people move their arms or legs.

Using the app, Shion&#8217;s teachers compared Shion&#8217;s heart rate response when they played different musical instruments. As they did this, they learned that her favorite musical instrument is the bell. After this, Shion was able to select her instrument for the first time.

Inspired by this finding, Mr. Uchida set out to monitor Shion&#8217;s heart rate in response to different activities in class. Using a heart rate monitor, an iPhone and an iPad, he video recorded Shion and the heart rate monitor simultaneously before, during and after the activities. He then reviewed each frame of the video, recorded the heart rate values, and graphed the data. Excitingly, there was a correlation between Shion&#8217;s heart rate and the activity. In other words, the decrease in heart rate reflected her positive emotions and level of focus. 