Multisensory and assistive technology solutions

Hämeen ammattikorkeakoulu
Woman trying glasses on.

Multisensory and assistive technology solutions

MATEC -research group focuses on multisensory and assistive technology solutions and innovation that promote learning and occupation.

The Group has currently three subgroups (i) multisensory wayfinding solutions in developing smart accessible environments; (ii) accessible learning and occupation with mobile based assistive technology and IoT, (iii) understanding of how multisensory features affect human performance on user-interfaces and accessibility.

The goal is to create smart assistive technology solutions and innovations that emphasize multisensory features and are based on real life needs, with the objective of being able to guide the future design and use of high-performance and high-efficiency accessible devices needed in learning and occupation.

The Group conducts trans-disciplinary research that is embedded on Triple or Fourth Helix –model, takes place on real-life environments, test beds and Living Labs, and develop solutions co-creatively in innovation ecosystem way. Our group’s vision is that people who need assistive technology should be able to experience the solution development as creators, not just as consumers. Lessons learned by this collaboration are useful for all actors.

Solving way-finding challenges of a visually impaired person in a shopping mall by strengthening landmarks recognisability with iBeacons

Because of reduced visual capacity, mobility and orientation are very challenging for blind and VI people. In addition to VI people, mobility challenges in large complex buildings concerns almost everyone. Accessible design aims to ensure that spaces, applications and contents are accessible for everyone, including people with impaired perceptual capabilities.

Read more

Projects

MATEC’s research projects are built on a strong ecosystem collaboration with end users and entrepreneurs. Assistive technology is developed to real end-user needs in cooperation with companies. Key projects:

Promoting learning and development in multisensory environments with assistive technology solutions (SPEDUTEC)

There is large body of research on the topic of technology to support learners with special educational needs. Application possibilities look promising. However, several issues need further attention: there is a need to advance co-creation and collaboration with end-users of product and service design; to develop interdisciplinary approach and cross-disciplinary communication, especially in regards access to learning and technology, and social aspects (e.g. inclusion and social interaction, independence and self-confidence), and ; to develop competence among professionals working in education, health and social sector, as well as among teachers in higher education in these areas. Our Nordic-Baltic project-cooperation between five countries and eight partners, within an activity framework of four workshops and case-studies, shall address these areas. The project will utilize broadly social media and video-conferencing in its work, and in internal and external communication. Iterative process-evaluation will be applied and outcomes reported as a multimedia-publication.

The project goal is two-fold: (i) to increase our understanding on theory and pedagogical and technological practice on learning through assistive technologies (ATs) in multisensory environments (MSE) by sharing our knowledge and experiences, and (ii) to build a new Nordic-Baltic network on the topic. Special education is the disciplinary focus in our project. Thus, our work will ultimately support education as a human right for all people (UN General Assembly, 2007) and promote EU’s goals for ICT for learning and inclusion. ATs refer to “any product or service design to enable independence for disabled and older people” (FAST, 2001).​​

The specific goals are:

  1. To learn from sharing our knowledge and experiences on what kind of, by whom and how are multisensory environments and assistive technology solutions practiced, organized and developed in Nordic and Baltic countries in order to promote learning among disabled people of all ages.
  2. To develop a multimedia product to wider public based on the main activities: the four Nordic-Baltic case studies.
  3. To establish a new cross-sector Nordic-Baltic network around the topic.
  4. To promote the cultural and social community and understanding across the Nordic-Baltic region.

The project topic is a new complex area, which is rapidly developing, but where there are big issues and challenges that are common across the participating countries and that sorely need more attention. Thus, the involved partners from difference sectors and disciplines share the concern for these issues in their efforts to promote lifelong learning in different contexts and settings, and among different learners. Cross-country learning, besides cross-sectorial learning, will be accentuated by the fact that some of the participating countries have both better access to and have, thus, gained more experience from practice within the topic area.

Keywords: Assistive technology, Multisensory environments, Nordic-Baltic case studies, SPEDUTEC

Partner institutions:

  • Stord/Haugesund University College (NO)
  • Lund University, Faculty of Engineering (SE-LTH)
  • Finnish Association on Intellectual and Developmental Disabilities (FI-FAIDD)
  • University of Tallinn (EE)
  • Via University College (DK)
  • Guldhornet (DK)
  • Häme University of Applied Sciences, Education and Research Center for Wellbeing (FI)
  • Eldorado (SE)

Project period: 01.08.2014 – 31.10.2016

Funding: Nord Plus Horizontal

Improving independent mobility way-finding solutions (ILSI) I

According to WHO (2011) there are approximately 285 million visually impaired (VI) persons worldwide. 30 million blind or partially sighted persons living in Europe, and 80000 of them in Finland (Ojamo 2014).  Mobility and orientation are very challenging for the blind and visually impaired persons. Reduced visual capacity challenges persons’ every day’s spatial problem solving in several ways, e.g. how to receive, perceive, interpret, and process information needed in intercourse with the physical environment. Difficulties in way-finding may cause stress and anxiety, which may lead to situation where blind or visually impaired person avoids leaving home to visit unknown, complex places and large-scale spatial environments, like shopping centres, unless having assistance. Way-finding refers to how a person orientates and navigates through an area or a space. The way-finding instructional information is mostly presented in a visual form, which means it is not accessible for blind or visually impaired people. It is hard for a blind person ​to gain a non-visual overview of the new place, and to map its landmarks.

Purpose of the ILSI -project was to develop smart multi-sensory way-finding solutions that will 1) easy VI person’s independent transactions and way-finding indoors, and 2) increase independent mobility and travelling with public transport. The aim was to identify and pilot iBeacons’ potential for exploitation of multi-sensory and mobility to facilitate the tracking, routing and mapping solutions. iBeacons’ potentials and challenges for were piloted at the Itis shopping center inside and in the context of buses, subways, bus and rail platforms at both signalling tables.

As results of the project, new kind of indoor navigation model was developed. The basis of the model is on landmarks recognised by blind people. These landmarks recognisabilty were strengthen with iBeacons mediated three level messages.

Solving way​­‐​finding cha​​llenges of a visually impaired person in a shopping mall by strengthening landmarks recognisability with iBeacons​​​

The results will be utilised in the development of intelligent transport environment and accessible guidance systems to serve all customers. Smooth stress-free travel-chains, the mobility environment’s multi-sensory intelligence and safety improves everyone’s, and especially the visually impaired people’s independent mobility and movement comfort.

Project partners: 

  • Finnish Federation of Visually Impaired
  • Kanta-Hämeen Näkövammaiset ry
  • Liikkumistaito Polku H. Hirn
  • MIPsoft Oy
  • Häme University of Applied Sciences
  • Itis Shopping Centre

Project period: 15.1.2014-31.12.2015

Improving indipendent mobility with multimodal way-finding solutions II (M-Way Dixi and M-Way Alepa)

Mobility and orientation are very challenging for the blind and visually impaired persons. Reduced visual capacity challenges persons’ everyday’s spatial problem solving in several ways, e.g. how to receive, perceive, interpret, and process information needed in intercourse with the physical environment.

Project partners at the case of M-Way Dixi project:

  • Avaava Oy
  • Versoteq Oy
  • Liikkumistaito Polku H. Hirn
  • MIPsoft Oy
  • Häme University of Applied Sciences

Project period: 1.8.2015-31.12.2017

Funding: Liikennevirasto, HSL, HKL

Project partners at the case of M-Way Alepa project:

  • SOK, Alepa Eliel
  • Häme University of Applied Sciences
  • MIPsoft Oy
  • Visually Impaired people’s Society of Kanta-Häme and of Uusimaa
  • Finnish Federation of Visually Impaired

Project period: 15.12.2015-31.12.2017

Funding: SOK, Alepa

Compet ence is the key (ORA)

 

Lately the unemployment of people with higher education has been increasing dramatically in Finland. The Häme University of Applied Sciences must meet this challenge and develop new guidance methods and procedures to support the employment of young people. This project will take the students and recent graduates with most difficulties in finding employment as the starting point, and later apply the methods to a larger student population. Students and graduates with disabilities or other special needs have a constant challenge in finding traineeship opportunities during studies and jobs after graduation, which exposes them to a high risk to permanently depend on retirement benefits. Supporting their employment requires new skills and procedures from the staff of the university.

The purpose of this project is to promote the employment opportunities of students and graduates with disabilities and special needs by developing:

  • holistic guidance services, and
  • digital tools, multimodal portfolios, assistive technology,
  • by strengthening the skills of the students,
  • by developing methods to better display those skills to potential employers.

This work will be done in cooperation with university staff, students, employers and   ​disability organisations.​ During the project, there will be collaborative training for the university staff in how to support students with special needs in seeking employment. Job coaches will provide help to students in finding traineeship placements and recent graduates in finding jobs. The project will also see​k new ways to connect employers and students and graduates with special needs. Special workshops will be founded for students to work in customer projects, to develop new assisting technology, digital tools and to create a new type of multimodal portfolio of skills. At the final phase of the project, the various methods and learnings will be applied to all the students with special challenges in finding employment, where applicable. As a result, the employment prospects of the graduates will improve.

Read more from the pr​​oject’s official webpages: Osaaminen ratkaisee

Project partners:

  • Vammaisten lasten säätiö ry (Vamlas Foundation)
  • Häme University of Applied Sciences

Project period: 1.5.2015-30.11.2017

Funding: European Social Fund

Activities:

Two learning environments were established:

  1. Assistive Technology Workshop (in Finnish) and
  2. Career Workshop (in Finnish)

COMPETENCE_IS_THE_KEY_Abstract_HAMK_EN.docx

Pathways - Personal Digil earning

  poluttamo logo

​​​Project develops practical means and models to help young people in transition phases find their way forward. The project idea involves support throughout studies and the gathering of knowledge. The project will construct functionalities to boost the process by using learners’ digital footprints in various educational institution’s systems. Through learning analytics​ we can make use of learners’ digital footprints e.g. anticipating support needs, clarifying gender differences and improving the competences of special needs groups such as disabled people.

​​Technology already allows us to design various forms of support so that they interest and motivate young people. We will focus ​on 1) developing personal study plans more illustrative, easily understandable and operable personal learning management tools.​ 2) Packaging Digital technologies and methods, such as we find in ePerusteet (“eFundamentals”), and e-portfolios in various educational institutions, into a dynamic whole that helps young people manage their learning. ​3) We also aim to clarify the interrelation of studies and career paths.​

Read more from the project’s official webpages: www.poluttamo.fi​

Project partners:​

  • The Association of Finnish eLearning Centre (Coordinator
  • Otava Folk Highschool
  • Omnia – The Joint Authority of Education in the Espoo Region
  • Educational Association of Citizens’ Forum SKAF
  • Häme University of Applied Sciences​
  • Centre for Open Systems and Solutions COSS
  • Tavastia Education Consortium
  • Lappland University of Applied Sciences

Project period: 1.11.2015-31.10.2018

Funding: European Social Fund​

Articles:

  • Kerkola, M. 2017. Learning analytics helping study advisor. SeOppi 2/2017.
  • Saarela, M. 2016. Multimodality and learning: Increasing understandability and accessibility. SeOppi 2/2016.
  • Seppänen, L. 2017. Challenges for learning analytics. SeOppi 2/2017.
  • Seppänen, L. 2016. Learning analytics call out for action. SeOppi 2/2016.

Grapewine f or Accessibility (KUPS!)

As a part of Culture for All Act in Finland, The Education and Culture Ministry’s recommendation is that public officers should evaluate public services’ accessibility in every 3 to 5 years. An accessible cultural site offers everyone an opportunity to take part and to gain new experiences. A welcoming, open attitude to diversity and awareness of diverse audiences is required during all stages of service planning, financing and production. Despite the recommendation, those who mostly need information on accessibility complain that the information is hard to find when needed.

KUPS -project focused on identifying, producing and sharing culture and leisure services’ accessibility information communally by using smartphones. The purpose of the pilot was to empower people with disabilities to produce and evaluate services’ accessibility information by using smartphones and apps. The project made possible for the community of people with disabilities to produce and share openly the services’ accessibility information needed.

The pilot was created mainly around the power of social-media perspective 1) participation and communally produced information, 2) using smartphones and mobile Apps, 3) crowdsourcing, 4) interactivity and co-creation, and 5) service design methods.

The pilot took place in Finland, Hämeenlinna, at the Verkatehdas Culture and Congress Centre. The mostly used apps were Google, Foursquare, Blindsquare, MBraille, Tap tap see, Dragon dictation.

In a three months KUPS -pilot (15.8.-31.12.2013) 3 blind and 3 physically disabled people learned to use smartphones (iPhone5), apps and social media in meaningful way, and produced cultural services’ accessibility tips. Tips were visible and accessible to all service users. Other users, including service providers, could interact and comment on written tip, or create tip of their own. Based on our findings, this method and the model are efficient, easy to use and complements conventional accessibility evaluation methods.

After the pilot project was done, the model has been developed further to count tips provider’s disability and used aid. Profile ex example could be like I’m blind person using white cane. With profile the accessibility tips are more useful among disability community. Currently the model is called Grapewine for Accessibility (G4A). The model competes at Nordic level’s Nordic Innovation Independent Challenge.

​Keywords: Accessibility, Mobile Apps, Co-creation, visual impairment, mobile impairment, service’s accessibility, crowed-sourcing

Project partners: 

  • Verkatehdas Culture and Congress Centre, Hämeenlinna
  • Hämeenlinna Art Museum
  • BioRex Cinema, Hämeenlinna
  • Finnish Federation of Visually Impaired
  • Kanta-Hämeen Näkövammaiset ry
  • Kanta-Hämeen Invalidit ry
  • Elisa Oyj
  • Hämeenlinnan Settlementti ry
  • MIPsoft Oy

Project period: 15.8.2013-31.12.2014

Funding: The Finnish Education and Culture Ministry​

Demonstrating the power of crowdsourced accessibility feedback with G4A

SPEDUCULT-project

The purpose of the SPEDUCULT-project is to strengthen and expand the SPEDUTEC-network, built in a previous NPHZ project, with the focus on added cultural value, on the one hand, and by further linking it with national and international level multisensory special interest societies and networks, on the other hand. ‘Strengthening’ here refers to increasing our knowledge and understanding of various sociocultural aspects of multisensory environments’ (MSEs) framework in the culturally versatile Nordic and Baltic countries.

Multi-Sensory Environment (MSE) can briefly be defined as designated and controlled space designed to alert or calm the person’s senses in order to enhance several purposes like playing, learning, health promotion, rehabilitation and over all increasing quality of life, feeling better and happier. Multisensory framework is part of methodological and practical studies within special education and social studies in higher education. In order to develop forward studies and practices of MSEs’ framework in Nordic and Baltic countries, more researched and documented knowledge, shared experiences, and cross-sectoral networking is needed. Our Nordic-Baltic project co-operation between five countries and nine partner institutions, within an activity framework of five seminars, workshops and survey, shall address these areas.

Activities comprise of the survey of the current MSE’s culture (education, methods & practises), exchange of experiences, and dissemination of the results. The project is also looking for best practices of applied MSEs to provide new practical ideas to design arousal and calming sensory activity. Outcomes of project will be presented at national and international conferences and as a multimedia report.

The SPEDUCULT aims are:

  1. To survey current MSE-cultures on a Nordic-Baltic Level, in order to improve quality and innovation of studies and practices of MSE in systems for lifelong learning: in higher education, in adult education and in special education at primary school in the participating countries.
  2. To provide new practical ideas for designing sensory activity of arousal and calming with best practices of applied MSEs.
  3. To strengthen and expand the SPEDUTEC Nordplus Horizontal network with new partners, institutions and organisations.
  4. To disseminate up-to-date MSE knowledge and practices for a broader audience interested in MSEs in the Nordic and Baltic countries.

Keywords: Multisensory environments, Nordic-Baltic case studies, SPEDUCULT, SPEDUTEC

Partner institutions:

  • Häme University of Applied Sciences, Education and Research Center for Wellbeing (FI-HAMK) (coordinator)
  • The Oslo School of Architecture and Design (NO-AHO)
  • Lund University, Certec (SE-LU)
  • City of Helsinki, Department of Social and Health Care, Services for the Disabled (FI-Services for the Disabled)
  • Käo Basic School (EE-Käo Basic School)
  • Via University College (DK-VIA)
  • ISNA-mse International snoezelen association-multi sensory environment (DK-ISNA-mse)
  • City of Gothenburg, Eldorado (SE-Eldorado)
  • NTNU- Norwegian University of Science and Technology (NO-NTNU)

Project period: 01.09.2017 – 31.12.2019

Funding: Nord Plus Horizontal​

Activities:

Core Facilities

MATEC Research Group conducts research on real-life environments, test beds and Living Labs. Our Research Equipments allow data collection in these real-life situations.

Itis Shopping Center and Itäkeskus Metro Station test bed

Tikkurila Travel Center test bed

Eliel Alepa test bed

Sensomotoric Instruments (SMI) and Emotiv Epoc Headset

In collecting data, we have Sensomotoric Instruments’ (SMI) Eye Tracking Glasses 2 Wireless (SMI ETG 2w), which are designed to record a person’s natural gaze behaviour in real-time.​ These Glasses allow binocular tracking up to a 120 Hz sampling rate over the whole trackable field of view. Compined with a high definition scene camera and automatic parallax compensation this ensures robust and accurate data over all distances.

Along with the SMI we collect biosignals by using the Emotiv Epoc. Emotiv Epoc is a Brain Computer Interface and Scientific Contextual EEG offering high resolution, 14 EEG channels + 2 references. A wireless EEG system for research enabling entertainment, market research & usability testing and neurotherapy.​

Go Pro cameras

We have severals Go Pro cameras to collect data.

Research group members

Our research is user-oriented and trans-disciplinary nature.

Merja Saarela, Ed.D. (Special Education), principal lecturer (research) and Research Group Leader in MATEC

Saarela is specialised on social-neurocognitive learning difficulties (especially learning difficulties caused by developmental disabilities, dementia, sensory deficits), interaction, learning and knowledge assessments, education evaluation.​ Her research interest are in user-centered wellbeing and education technology, assistive technology, gaming, action and interaction platforms​.

Read more: https://fi.linkedin.com/in/merja-saarela-95285614

Pirjo Seddiki, D.A., principal lecturer

Seddiki has specialised on aesthetic design. She applies aesthetics and artistic research methods to design processes.

Read more: https://www.linkedin.com/in/pirjo-seddiki-ab5b5755/

Lasse Seppänen, Lic.Sc. (Tech.), principal lecturer

Seppänen has over 16 years experience as senior lecturer at HAMK University of Applied Sciences. Currently his main RDI interests are in eLearning and Learning analytics.

Read more: https://fi.linkedin.com/in/lasseseppanen

Mika Virolainen, B.Sc. (Information Technology)

Mika Virolainen is a research assistant at HAMK MATEC. His specialities are in software coding, 3D-games and simulations.

Read more: https://fi.linkedin.com/in/mika-virolainen-348b0b105

Tarjaleena Tuukkanen, senior lecturer

Tarjaleena Tuukkanen has several years interest in digitality, e-learning in general and assistive technology. At HAMK she’s been developing web learning solutions with teaching staff and IT administration. Her specialities are in PLEs, Google in Education, ePortfolios, Social media and its applications in education and at work, O365, Kaltura. Currently she is working in Competence is the key -project as an assistive technology specialist. She runs special workshops to develop new assisting technology, digital tools and to create a new type of multimodal portfolio of skills for students with special needs.

Read more: https://fi.linkedin.com/in/tarjaleena

Maija Apunen, Music technician, Sudent assistant

Apunen has specialised on music technology and ICT, Eye Tracking Glasses (SensoMotoric Instruments) and EEG headset (Emotiv).

Read more: https://www.linkedin.com/in/maija-apunen-439152106/

Video: BlindSquare Indoor

MATEC in social media

Wow! Hieno innovaatio Selkokeskukselta: ”Selkokielen mittarin avulla voidaan arvioida, onko jokin teksti selkokieltä vai ei.” Lue lisää: https://t.co/pO9zkAOs3g

Watch why captioning videos helps so many of us: https://t.co/VSS5tJUtTA

I had a very exciting morning today. I took part of experiencing automatically driving car at Vantaa, Kivistö, Finland.

Currently, we're getting glimpses of the future world and business, And there's so much to learn and innovate. Want to learn what's happening with IoT and Blockchain? Here's one chance for you, take a look:

Currently, we're getting glimpses of the future world and business, And there's so much to learn and innovate. Want to learn what's happening with IoT and Blockchain? Here's one chance for you, take a look:
Equinix Sweden@EquinixSE

Du har väl inte missat vårt kommande event i Stockholm den 17 Maj? Temat för årets event är de heta ämnena IoT och Blockchain. Anmäl dig här: https://t.co/Rz27ReT3sx

Welldone Google! Read more about ”Wheelchair accessible” routes in transit navigation @google https://t.co/adaSjDZe2h

#SUOMI100 (Säv. & San. Joel Puranen, 2017) https://t.co/DLu3AzjUKJ lähteestä @YouTube

Excellent article...Accessibility First: Rethinking the Way We Approach Website Design and Development https://t.co/CK8gE6KpZe

Great #Speducult Kick off meeting in Oslo, Norway 21. - 23.11.2017. Thank you #AHO & Birgitta and #NTNU & Anders.

Intohimoiselta saavutettavuuden puolesta puhujalta Ronja Ojalta löytyy selkeitä vinkkejä saavutettavampaan viestintään. https://t.co/GWQTvgW7tR

#Speducult project today in Oslo, Norway: Assistive Technology with musical instruments at the Dissimilis.no presented by Line and Christian. And later on calming environments at Emma Sansehus (emmagjestehus.no) presented by Katrine.

#Speducult project is visiting #NaCuHel.no today. Nature is a source for creativity, culture and health.

HÄME UNIVERSITY OF APPLIED SCIENCES