ÇEVRESEL ZEKA VE ALGILAMA DESTEKLİ AKILLI EV Welcome to all of them Halil Ertan, Hande Alemdar, Özlem Durmaz İncel, Cem Ersoy Bilgisayar Mühendisliği Bölümü, Boğaziçi Üniversitesi, İstanbul, TÜRKİYE

İÇERİK Giriş Çevresel Zeka ve Algılama Destekli Akıllı Ev Sisteme Bakış Sistemde Kullanılan Çevresel Algılayıcılar Veri Toplama Süreci Veriden Çıkarımlar Sonuç Here the content, it consists of introduction, initial studies, designinng a wsn in a real house, processing of the data collected from that house, and finally conclusuion and future works

Algılayıcı ağ teknolojilerindeki son gelişmeler GİRİŞ Yaşlanan toplumlarda ev ortamında insan hareketi izleme ve anlamaya yarayan sistemler Algılayıcı ağ teknolojilerindeki son gelişmeler Nasıl izliyor? Hangi algılayıcılar? More invesment is required for elderly care especially for ageing societies. Besides, shortcoming in skilled caregivers is another problem. Monitoring and recognizing the ADLs of elderly by speacial systems remotely, daily routines and habits of people can be understood, and health officers can be informed in advance for the probable disease. This will enable the disabled people to live independently, and improve the life quality of them. One question arise at this point? Which kind of systems perform this? Accounting the recent advances in sensor network tech, WSNs are ideal platform for this purpose. Different types of sensors can be utilized for this aim, like cameras, microphons, on-body sensors, or ambient snesors, all of them have its own advantage/disadvantages. Cameras do not take into consideration the privacy issues mostly, and it is obivous that on-body sesnros limit the movements of people. Compeared to others, ambient snesors are much more utilized recently. And lastly, there are challenges stemming both from network side, human activity recogniiton side. For wsn, homes are typically multi path environment due to the walls and furnitures in the house. Battery lifetime is another important concern. They have limited ability for operating complex software mostly. For human act. Rec. Side, we can consider that most ADLs consists of its sub componenets, activities, and people can perform the activities in different ways mostly. Farklı tipte algılayıcılar: kameralar, mikrofonlar, insan vücudu üzerindeki algılayıcılar, çevresel algılayıcılar

Hangi Algılayıcılar Tercih Edilmeli ? - Kameralar Özel hayat? - Vücuda Bağlı Algılayıcılar Hareketleri kısıtlama? - Çevresel Algılayıcılar

Algılayıcı ağ teknolojilerindeki son gelişmeler GİRİŞ Yaşlanan toplumlarda ev ortamında insan hareketi izleme ve anlamaya yarayan sistemler Algılayıcı ağ teknolojilerindeki son gelişmeler Nasıl izliyor? Hangi algılayıcılar? More invesment is required for elderly care especially for ageing societies. Besides, shortcoming in skilled caregivers is another problem. Monitoring and recognizing the ADLs of elderly by speacial systems remotely, daily routines and habits of people can be understood, and health officers can be informed in advance for the probable disease. This will enable the disabled people to live independently, and improve the life quality of them. One question arise at this point? Which kind of systems perform this? Accounting the recent advances in sensor network tech, WSNs are ideal platform for this purpose. Different types of sensors can be utilized for this aim, like cameras, microphons, on-body sensors, or ambient snesors, all of them have its own advantage/disadvantages. Cameras do not take into consideration the privacy issues mostly, and it is obivous that on-body sesnros limit the movements of people. Compeared to others, ambient snesors are much more utilized recently. And lastly, there are challenges stemming both from network side, human activity recogniiton side. For wsn, homes are typically multi path environment due to the walls and furnitures in the house. Battery lifetime is another important concern. They have limited ability for operating complex software mostly. For human act. Rec. Side, we can consider that most ADLs consists of its sub componenets, activities, and people can perform the activities in different ways mostly. Kablosuz ağ tarafından kaynaklı ve insan hareketi algılama tarafından kaynaklı zorluklar Farklı tipte algılayıcılar: kameralar, mikrofonlar, insan vücudu üzerindeki algılayıcılar, çevresel algılayıcılar Zorluklar?

İnsan Hareketi Tanıma Uygulamalarının Kullanım Alanları Çevresel Destekli Yaşama Uzun vadede insanların günlük rutinlerini ve alışkanlıklarını öğrenip, anormal durumların önceden tespiti Sağlık Hizmetleri Düşme İlaç kullanımını hatırlatma Akıllı Ev Aşırı su elektrik tüketimini algılama İkna Edici Uygunsuz davranışları değiştirme These are the main fields for HaR systems. As mentioend before, in AAL, learning the daily routines of residents in the long term and detecting abnormal situatiions are aimed. In healtcare domain, falls are consdired mostly which is an emergency situation,and can be detected in the short term monitoring. Another important field is smart home applications which measures for example the usage of lectrictiy and water and inform the residents in the situation of excessive usage to serve for a greener world. And there are also some applicaitons which induce the behaviour change of residents.

ÇEVRESEL ZEKA VE ALGILAMA DESTEKLİ AKILLI EV Sisteme Önbakış Arduino Fio platformu & Xbee alıcı/verici 7 farklı tipte 20 farklı çevresel algılayıcı 2 Kişisel Alan Ağı 10 Hz örneklem aralığı 27 farklı aktivite 30 tam gün boyunca 2 insanın hareketlerinin izlenmesi And we came to the ur mian stıdy. Firstl, let me introduce the system properitis. We utilized aruino plt. And xbee modues like the previous study. During the data collectişon process, we used 20 ambinet sensors with 7 different types. Since we could not cover the enitre house with juset one bs, we used 2 different bs , we selct sampling rate as 10hz, concern a large scale of activities 27 activites. And monitor and collect adls of two residents for 30 full days. These are the … Sistemde kullanılan farklı çevresel algılayıcılar

Evin Krokisi Sistemdeki İki Kişisel Alan Ağının Kapsama Alanı The setup of the house is depicted. It is approximately 50 m2. The coverage are of the Pans can be seen. On eof the BS is in the kitchen the other on eis in the living room. Sistemdeki İki Kişisel Alan Ağının Kapsama Alanı

Sistemdeki Bilgi Akışı Here information flow in the system can be observed. When an event occurs which is the targeted activity for the sensor nodes like watching tv studying, perparing meal etc.Sensor nodes detects it and sends sensor values to the relevant BS by using Zigbee protocol. Which is the task of xbee modules in sensor nodes. Then received sensor values are transmitted to a Pc via serial communication. After proccesing of the data in PC, sensor readings like in this format occurs. Sistemdeki Bilgi Akışı

Baz İstasyonları Mutfakta ve oturma odasında 2 ayrı baz istasyonu Kişisel bilgisayara bağlı bir Xbee alıcı/vericiden oluşuyor Yerleştirirken algılayıcılarla olan görüş çizgisine dikkat edilmeli There are 2 bs in the system as mentioned before, one of them is in the kitchen like in the figure, and the other one is in the living room. A BS concsits of Xbee module connected to a PC. While placing the BS, line of sight with the sensors should be taken in to consideration/accounted. We have an interface in the PCs which receive synchronize and process the collected data. We have a limited time I think, I just want to show you the pictures of the sensors deployed in the house very quickly, if you have any questions spesific to any sensors, we can discuss it after the presentation. Mutfaktaki Baz İstasyonu

Fotoseller En basit ve en sağlam algılayıcılar There are 6 photocells in the system. Inside the wardrobe of the residents, bathroom cabinet, in refiregerator, and in kitchen drawer. They have a simple working principle, they sense the light in the environment, and give different values in change of light amount in the environment. Like opening and closing a wardrobe changes the light amount inside the wardrobe. We can say taht they are the most robust and user firendly sensors. And their duty cycle is very low considering that the door of the refiregetor is kept as open at most 15-20 minutes during a day. En basit ve en sağlam algılayıcılar Çevredeki ışık miktarındaki değişiklikleri algılıyor Gün içinde en az aktif olan algılayıcılar

Kuvvet Algılayıcıları Twe used 3 FSR. Two of them share the same Fio as seen in the first figıre. They are placed under the leg of couches in living room, and bed in the bedrrom. If we consider the sleepigng durations of people, we can simply say that theay re the most active sensors in the system. We used dynamic thresholds for FSRs , which redetermine the threshold value when it is resetted. And as alerned lessons, it should nopt be prefferd to be used with together moveable appliances like chair. En aktif algılayıcılar Dinamik baraj seviyesi Hareketli mobilyalarla kullanılmamalı

Sonar Uzaklık Algılayıcılar Actaluu they are hte most advanced sensors in the system, which can indicate the distance of the objects. However, we used them in our case, for a more simpler purpose whether someone passes in front of tit or not. The reason of most of the false positive and neggtive situations is the subtle change in the location. En ileri seviye algılayıcılar 6.45 m menzil Yerinden çok ufak sapmalar yanlış sonuçlara sebebiyet verir

Kısa Mesafe Uzaklık Algılayıcılar Çok küçük menzil 10 cm Klozet kapağı üstünde Banyoda musluk üstünde Oturma odasında sandalyenin arkasında They are another kind of distance sensors. They have a very samlle range like 10 cms. They are placed …. Respectively.

Kontak Algılayıcılar Magnet ile beraber çalışır Çok hassas ve kırılgan algılayıcılar Kapıların açılıp kapatılmasını tespit etme amaçlı En problemli algılayıcılar They work with a magnet together and have avery simple working principle. While using them, we aim to detect the opening and closing of the doors. If we consider that they are very sensitive and fragiel, we can conclude that they are the most problematic sensors in the system. İf the contact sensor and magnet is very close to each other, the current is conveyed, otherwise it is not.

Sıcaklık Algılayıcı Mutfakta ocak üstünde Çevre sıcaklığını ölçer 33.8 centigrade baraj seviyesi Aktivite sonlandıktan sonra çevre hemen soğumuyor There was only one temp sensor in the sytem which is placed above the oven in the kitchen in order to sense the temp change in the environment which generally occurs while preparing meal. We considerd the entire data during summer, and for this reason determine the threshold value as appor. 33 degree. One shortcoming about it is that environment is not cooling down immediately, for this reason, it still continue firing for a while at most 3-4 minutes. After prpering meal is over.

Kızıl Ötesi Algılayıcı TV altı Uzaktan kullanma durumunu tespit etme 100 Hz ile algılama 10 Hz ile gönderim IR is placed under the TV in order to making inferences about watching TV actvites of residents. It basically dtects the remote controller usage of the residents. Since pushing a button of the remote controllr is a much more faster activity compared to the other concerned activities in the system, we sense the nevironmetn with 100 Hz in order to not miss any data, and transmit again with 10 Hz.

Veri Toplama Süreci Algılayıcı Aktivite Eşleştirmesi Tip Algılayıcı Yer Amaçlanan Aktivite Kontak Algılayıcı Co1 Dış kapı Dışarı çıkma, misafir kabul etme Co2 Banyo kapısı Duş alma, tuvalet Co3 Duşa kabin Duş alma Kısa Mesafe Uzaklık Algılayıcı Di1 Oturma odasındaki sandalye arkası Sandalyede oturma Di2 Di3 Banyoda musluğun üstü Elleri yıkama Di4 Banyoda klozet kapağı Tuvalet alışkanlıkları Sonar Uzaklık Algılayıcı So1 Antrede duvarı Antre boyunca geçişler So2 Mutfak duvarı Mutfak aktiviteleri Kuvvet Algılayıcı Fo1 Koltuk altında Koltukta oturma, uzanma, uyuklama Fo2 Fo3 Yatak altında İkinci kişinin uyuması Kızıl Ötesi Algılayıcı Ir1 TV altında TV izleme Fotosel Ph1 Giysi dolabı içinde İlk kişinin giyinmesi Ph2 Koltuk içinde İlk kişinin uyuması Ph3 Buzdolabı içinde Buzdolabı kullanımı Ph4 Mutfak çekmecesi içinde Ph5 İkinci kişinin giyinmesi Ph6 Banyo dolabı içinde Banyo aktivitesi Sıcaklık Algılayıcı Te1 Mutfakta ocağın üstünde Yemek pişirme, yemek yeme Sensor activity matching was the first parameter we take into consideration during data collection process, especially for selecting and palcing the sensors. For instance, temperaure sensor is used for detecting cooking having meal activites, sonar distance sensors aim to detect passing along the hall. FSrs are used to learn sleeping habits of residents.

Veri Toplama Süreci Pillerin Ömürleri 1800 mAh lipo piiler Örnekleme Aralığı 10 Hz 100 Hz Boş Mod Aktif Kalma Oranı 10% 24h19m 24h11m 50% 24h29m 23h57m 90% 24h10m 24h02m Uyku Modu Aktif Kalma Oranı 145h21m 146h43m 49h23m 45h32m 28h30m 27h07m We performed a test about life time of the batteirs, and the rusults can be seen. In ıdle mode wireless comm. Mod. Are always active, and in sleepin gmode, they sleep when there is not an event arıund. In order to give an insgiht about duty cycle of the sesnors in the system, phtocells …., and FSRs…. As it can be observed, there is a significant increse in…… Pil Ömürleri Deney Sonuçları Fotoseller %10’dan daha düşük bir aktif kalma oranına sahip Kuvvet algılayıcıları %50 civarı bir aktif kalma oranına sahip Uyku modu kullanılarak pil ömürlerinde ciddi bir artış gözlenmektedir

Veri Toplama Süreci Etiketleme: Şu an ne yapıyorsun? Basit bir ara yüz kullanarak kendini etiketleme 27 farklı aktiviteyi kapsar Her gün ortalama 60-100 doğru etiket Another critical issue is the lebeling the activites. The residents perfomed it by the help of simple GUI as seen in Figure. IT is a very simpe and user firendly interface. The resident only selects who he is from the tabs, and then selcts the activity he is perfoming currently, and then only push the label data button. 27 differnt activite s are concerned, a very detailed set actually. We have obtianed labels between 60-100 accurate labels as an average for each day which is very valuable for such systems. Etiketlenmenin gerçekleştirildiği ara yüz

Algılayıcı Okumaları Rastgele bir günden algılayıcı okumaları Sensor readingds from a random day is depicted. As you see, some of the seneors like fsr s are much more active than the other sensors. Rastgele bir günden algılayıcı okumaları

Aktivite Verisi Senin Hakkında Ne Anlatabilir? İnsanların hayat standartlarını iyileştirmek Sağlık Sosyal İlişkiler (Birden çok kişiden aynı anda veri toplanması bu konuda bize olanak sağlıyor) Kişisel Bütçe (Daha temiz bir dünya) And the last step is the inference about the behaviours of residents. By using such human acitivty montiroing and recognition systems, promoting the life standards of people is aimed interms of mainly health, social interaction and personal finance in other words green world. %In oreder to give an insgiht, We took into account three factors in the inferences which shape the beahaviours of residents basically, …. Aktivite Süreleri Aktivite Zamanları Aktivite Sıklıkları

Aktivite Verisi Senin Hakkında Ne Anlatabilir? İki kişinin tuvalet süreleri

Aktivite Verisi Senin Hakkında Ne Anlatabilir? As an example, it is a good example, beacuse two residents have a very different slepp and napping habits within the day. For instance first person sleeps more than the second residents, on the other hand, secodn person closes the gap by longer napping withint he dya Kişilerin uyku ve uyuklama süreleri

Aktivite Verisi Senin Hakkında Ne Anlatabilir? Another example is about starting time of acitivites. In this figure, wee see startinf times of having measl for the first rewidents during 30 days. AS yo usee for some consecutive days, he skipped the lunch. Birinci kişinin gün içinde yemek öğünlerine başlama vakitleri

Aktivite Verisi Senin Hakkında Ne Anlatabilir? Since, we collected data from a house in which multiple residents live, this enable us to study the relations between the residents who share the same house. This figure shows the percentage of common acitivite durations within the day in the house except sleeping and going outside. IT is approximately 25 percent on average. Kişilerin evde gün içinde ortak yaptıkları aktivitelerin gün içinde evde geçirdikleri toplam süreye oranı (Uyuma ve dışarı çıkma hariç)

Sonuç Toplamda 7 farklı tipten oluşan 20 algılayıcının kullanıldığı bir sistem tasarımı Çevresel algılayıcılar Detaylı 27 aktivite kapsanmıştır Her gün ortalama 60-100 doğru etiket sayısı Pil ömürlerini test eden deneyler En önemlisi, iki kişinin 30 tam gün boyunca gerçek bir evde günlük aktivitelerinin izlenmesi Let me summarize our contributions in conclusion. We designed….. And utilized ambient sensors which is much more advantageous compared to cameras and on-body sensors. We concerned a very detailed set of acitivities comapred to most othr similar study, and obtain labels between 60-100 accurat labels for each day. Differnt appraches are utilized in order to model the collected data. And most importantly, we have collected ADLs of multiple residents in a real house for 30 full days.

Dinlediğiniz için teşekkür ederim. SORULAR ? halil.ertan@boun.edu.tr hande.ozgur@boun.edu.tr ozlem.durmaz@boun.edu.tr ersoy@boun.edu.tr http://www.netlab.boun.edu.tr/WiSe/