Modeling of the activity as a fuzzy-temporal concept

Modeling of the activity as a fuzzy-temporal concept

Literature review: data mining in the Smart Home

Introduction to research in ambient intelligence and Smart Homes

A major development in recent years is the importance given to research on ambient intelligence (Ami) in the context of activity recognition. Ami, in opposition to traditional computing where the desktop computer is the archetype, consists of a new approach based on the capacities of mobility and integration of digital systems in the physical environment, in accordance with ubiquitous computing [51]. In this new vision, processors, sensors and actuators are integrated to the objects that are applied every day in order to carry out the activities of daily living (ADLs). This issue allows the ADL objects to be embedded in the environment as unobtrusive, interconnected, adaptable, dynamic and intelligent. This mobility and fusion are made possible by the miniaturization and reduced power consumption of electronic components and the omnipresence of wireless networks. It allows foreseeing the opportune composition of devices and services of all kinds on an infrastructure characterized by a granular and variable geometry, endowed with faculties of capture, action, treatment, communication and interaction [22]. Therefore, Ami is the vision of a future in which environments support the people inhabiting them. The envisioned Ami environment is sensitive to the needs of its inhabitants, and capable of anticipating their needs and behavior [19]. It is aware of their personal requirements and preferences, and interacts with people in a user-friendly way. A majority of works in literature [54] on the subject identify Ami as an intelligent, embedded, digital environment that is sensitive and responsive to the presence of people. For most authors, the key features here are intelligence and embedding [55]. By the term embedding, we mean miniaturized devices that merge into the environment while an inhabitant performs his or her ADLs. By intelligence, we mean the system is sensitive to context, is adaptive, learns from the behavior of users and gives punctual required services at the right moment. This important feature constitutes the central focus of this thesis as we address the issue of intelligence, learning and activity recognition. Similarly, research in [56] identifies five related key technology features: embedded, context aware, personalized, adaptive and anticipatory. Once again, we note that most of the identified key features (personalized, adaptive and anticipatory) are related to the issues that we address in this thesis, which are learning and recognition.

The Ami concept comes with great expectations. Some of them are near to realization, and others look more like science fiction for now. But the scope of potential Ami applications is rich and vast. For instance, families may now never lose track of their Alzheimer’s disease-ridden relatives because of location sensors and miniature communication devices sewn into the fabric of clothes [55]. Blind people may be guided in unfamiliar environments by intelligent signposts and public transport timetables that may communicate via wireless headsets [57]. Traditional memory aids can remind the user about activities on their daily schedule, but more sophisticated memory aids can be context-sensitive. These can « observe » the user in his or her activities, infer desired tasks and, on that basis, issue reminders and guidance [21]. Today, we could even monitor walking patterns of cognitively-impaired people with smart shoes [58].

Application of Ami in the indoor context: Smart Homes

The potential of Ami at home has been the subject of research for at least a decade in both academic laboratories and major industries. For instance, the Microsoft Corporation Domestic Home [55] shows a case of Ami-based smart appliance technologies for residences. There are no desktop or laptop computers, but the wallpaper is interactive, and can be controlled by tablet PCs. The mailbox outside tracks the mailman’s location by using a GPS and users can get a realtime estimate of when mail will arrive, on the mailbox display or by cell phone. RFID (radio frequency identification) tags embedded into envelopes even provide information about what mail is on the way. RFID tags are active barcodes which attach digital information to objects. This technology is being increasingly used by industry for tracking inventory. These tags can be quite small and do not require a battery. In the Microsoft Home, there are RFID tags on clothes as well. In a bedroom, by holding clothes up to a mirror which doubles as a screen, one can get information about these, including whether matching items like a skirt or jacket are in the wardrobe or laundry. The kitchen has an intelligent note board. If you pin a pizza coupon on it, the restaurant’s menu and phone number are displayed. You can call the restaurant with a tap on the board.

In the UK, British Telecom (BT) and Liverpool City Council have run trails on telecare technology within a project devised by BT’s Pervasive ICT Research Centre [59]. The trial concerned a system that responded to crises in the home. Each home contained 12 wirelesssensors, including passive infrared sensors, entry switch sensors, occupancy sensors, a toilet flush sensor and a temperature sensor, all connected to a central gateway and a BT server via a secure broadband IP channel. If a cause for concern is flagged, a voice call is made to the home’s occupant. If he or she confirms that they are okay, voice recognition technology is used to cancel the concern, otherwise a voice alert is sent to selected personnel who can then use a standard Web browser to access information about the inhabitant and circumstances of the alert.

Another example refers to the work of [60], where the bathroom mirror can remind a person of the medication he or she has to take, and the car stereo can tune into the same station that was on during breakfast. Some ideas and functionalities are distant visions, such as self-monitoring and self-painting walls, and lighting and furniture recognizing emotions and moods; some have already produced prototypes such as dormitories that learn the simple preferences of their single occupants regarding open or shut windows and level of lighting or heating. Some simple Amibased devices such as temperature sensitive heating systems, movement-sensitive lighting and light-sensitive blinds are even routinely available in stores.

Smart Homes

Various names have been used to describe homes equipped with pervasive technology to provide Ami services to the residents who inhabit these. Smart Homes may be the most popular term [48], and other terms include aware houses, intelligent homes, integrated environments or alive, interactive, responsive homes/environments. Innovation in domestic technology has long been driven and marketed by the desire to reduce labor and improve the quality of time spent at home. This continues to be one of the motivations for development of Ami at home. Other factors include technological advances and expectations, and an increasing trend in a way of life that blurs the boundaries between home, work and places of rest and entertainment. In this thesis, Smart Homes are foreseen as a potential way of assisting cognitively-impaired people in performing their activities of the daily living [12] at home, thus increasing their quality of life and optimizing spent energy, which are the desired goals of Smart Home design. Technically, such a Smart Home can be described as an ambient home-similar environment which, through its embedded sensors, captures accomplishment of actions concerning to the realization of ADLs. The data resulting from observation of activity realization in the Smart Home forms what we consider a big data warehouse. It must be analyzed by artificial intelligence techniques embedded within the Smart Home in order to provide information about home state normality and then infer required assistance for the Smart Home resident or achieve other Smart Home design objectives. Thus activity recognition applies artificial intelligence techniques in order to provide key knowledge for the Smart Home by interpretation of Smart Home observations, in order to make decisions for assistance of the Smart Home resident or to achieve any objective of Smart Home design.

Knowledge provision for reasoning

Knowledge is the set of rules that indicate relationships between data points. Knowledge is a basic and essential requirement in reasoning. Generally, the required knowledge for a typical reasoning system can be sourced in two main ways. The first way is to transfer the human expert idea and apply his idea as knowledge; but considering that there may be a lot of machine states or narrow and delicate rules in activity recognition, it would be a very difficult task for an expert to define the rules for recognition of activities. Moreover, the expert may not be available every moment that we need him or her. Sometimes, he or she may not be fast enough at this task for every Smart Home in the world. There would be many exceptions for this imaginary rule-base and the expert would have to consider many parameters and variables. For example, assuming that many activity features would be observed in a typical Smart Home (600 activity features are observed in LIARA), and most sensors indicate multiple states of their observations (rather than binary-state sensors), thus activity recognition in LIARA would deal with at least 2 =4.1495166+180 machine states. The reality is that this problem could cause billions of times more possible states, rather than the latter mentioned digit. Even the activity recognition problem is demonstrated as a sort of NP-Hard problem. It could become even more complicated if we desire personalized knowledge for each resident. The expert’s high-level rules should be translated into low-level rules or instructions in order to conduct reasoning and world actuation through machine facilities (in this case, the Smart Home). Furthermore, at the machine level, there are some delicate specific low-level rules (instructions) that an expert could not normally recognize or even express with his high-level knowledge. Although there are other limitations to an expert-driven approach, the mentioned argument is sufficient to not select &pure knowledgedriven approach as a practical solution for activity recognition in the Smart Home. The alternative solution, which solves knowledge-driven problems, is intelligent data understanding by inferring the hidden knowledge from this data.

Hence, by this second generic way, the data resulting from frequent observation of activity realization is mined. This operation is called data mining [26] and it refers to a group of artificial intelligence techniques that deal with knowledge discovery taken from data. Generally, discovered knowledge is kept in a knowledge base or rule base. The challenge of a pure datadriven approach is that even if the best data-understanding method is applied, many high-level rules or concepts (easily understandable for an expert) may not be discovered. The solution proposed by this thesis is not to completely eliminate the role of the expert’s knowledge from the knowledge provision process; however, this knowledge can be merged to the perceived rules from mined data through the fuzzy-logic theory [6, 61]. This theory provides facilities in order to correlate humanistic lingual knowledge to low-level instructions. Benefiting from this property on one hand, the expert could place the discovered.knowledge resulting from data mining in a fuzzy rule base (which holds fozzy rules) and, on the other hand, the expert can modify, remove or add the data-driven inferred fuzzy rules. Therefore, a fozzy rule plays the role of a sort of mediator between complicated high-level concepts (derived from expert’s idea) and the lowerlevel concepts that are understandable from data analysis.

Data mining

As it was mentioned earlier, knowledge is the set of rules that indicate relationships between data points. In order to automatically discover knowledge, data is mined. Generally, four operations may be conducted to discover relationships among data points in large databases. These are classification or prediction, clustering or segmentation, association rules discovery and frequent sequential pattern discovery [25]. The resulting knowledge, which is the set of rules, patterns or relationships, is stored in a knowledge base:
• Classification refers to an algorithmic procedure for assigning a given piece of input data into one of a given number of categories. In the classification process, stored data is used to locate data in predetermined groups. For example, a restaurant chain could mine customer purchase data to determine when customers visit and what they typically order.
• Clustering means the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. In the clustering process, data items are grouped according to logical relationships or preferences. For example, data can be mined to identify market segments or consumer affinities.
• Association rule learning concerns the methods applied for discovering relationships between variables in large databases. From a technical viewpoint, association rule mining searches for relationships between items in a data set. For example, in a sports store, Tshirts and trousers are mostly bought together (or in the same quantity). The sequence of transactions is not notified in this example. Another example is that 98% of people who purchase tires and auto accessories also buy automotive services. One other specification of association rule mining is that there are no restrictions on the number of items in the head or body of the rule.
• Sequential pattern discovery refers to the discovery of the frequent patterns that exist in a sequence of events. In this process, data is mined to anticipate behavior patterns and trends. For example, an outdoor equipment retailer could predict the likelihood of a backpack being purchased based on a consumer’s purchase of sleeping bags and hiking shoes. In this process, the sequence of transactions is annotated.

Basic data-mining operations (classification and clustering) can be conducted based on various parameters or logics; for example, regardless of existing data values in a data set, values of data points can be compared to an absolute value like a definitive integer and any number greater than it may be classified in a specific category, as well as classification of the smaller numbers. Another way is to find the average of the data values and to classify these based on the mean value. In conclusion, there are various ways to cluster or classify data points and so there are various data-mining algorithms which are pertinent to find the rules that explain data sets succinctly; however, the efficiency and functionality of each algorithm may differ.

Philosophy behind data-mining algorithms

The summarization of primary data into higher-level expressions is the principal task of data mining. A simple example of data mining is the equation that describes a line such as y = x. By a mathematical expression, all the points that exist by that equation are indicated in a summarized manner .

Quantitative strategy mining

Quantitative strategy mining can be summarized by the following law: if there is a rule, then we should observe a quantity of individuals, samples, objects, or concerning data that are caused by it; if not, it does not exist. The rule that produces more individuals is of higher priority. For example, if there is a car factory that produces cars, so we should observe its production on the street. If these cars are observed on the street, then there is a factory that produced them (rule). Through this viewpoint, we are able to count similar objects and, according to the quantity of objects, representative rules are indicated and ranked. Then, based on calculated statistics, probabilistic rules – which estimate the future states – are calculated. In this way, these probabilistic rules are called discovered knowledge.

Qualitative strategy mining

There is a second machine learning strategy which includes qualitative metho.ds of data understanding. These approaches work based on discovered similarities between objects. In these approaches, the possible number of discoverable relationships (rules) may be even greater than the number of objects. Qualitative approaches essentially presume that there is a plan behind events. By such strategies, possible plans that cause events in the world are modeled, and if observations match a part of a previously learned model, then it can be inferred that the plan is repeated. The law of this strategy says(C if there is a rule, then there is a perfect instance of such a rule and other objects of this rule are similar to this perfect object. The object that is the most similar to all of the other objects is selected as the best representative and indicates the generic rule’s specifications. Other objects that represent specifications of some other groups of individuals are labeled as instances of more particular rules. « 

General conclusion

In the proposed thesis, we presented a contribution to the field of artificial intelligence, following in the footsteps of temporal data-mining models and activity-recognition approaches. It allows us to take a step forward by providing answers to the issues raised in the introduction, which are related to providing a way to learn and recognize the activities of daily living in a Smart Home in order to be able to assist cognitively-impaired elders. More specifically, the thesis contribution takes the form of a new unsupervised temporal data-mining model for activity recognition, which allows anomaly detection and assistance provision in the Smart Home. In this model, the role of the expert in modeling, reasoning, data interpretation, inference making and recognition steps is decreased as much as possible; therefore, the proposed approaches of this thesis shed light in some directions for designing stand-alone Smart Homes. Therefore, it can be seen as a step toward the development of an assistive environment which is able to adapt itself to the resident’s profile and enable the possibility of independent living with respect to privacy through non-audiovisual observation and minimal need for an expert’s intervention.

Table des matières

1 Introduction 
1.1 Research context: aging population and the need for new technology
1.2 Smart Homes as a possible solution
1.3 Problem: data mining in the Smart Home
1A Summary of related works on temporal data mining in the Smart Home
1.5 Contributions of this thesis
1.6 Research methodology
1.7 Organization of the thesis
2 Literature review: data mining in the Smart Home 
2.1 Introduction to research in ambient intelligence and Smart Homes
2.1.1 Application of Ami in the indoor context: Smart Homes
2.2 Smart Homes
2.3 Knowledge provision for reasoning
2.4 Data mining
2.4.1 Philosophy behind data-mining algorithms
2.4.1.1 Quantitative strategy mining
2.4.1.2 Qualitative strategy mining
2.4.2 Temporal data mining
2.4.2.1 Identification of temporal datatype
2.4.2.2 Temporal data cleaning
2.4.2.3 Time-series similarity measures
2.4.2.3.1 Euclidean distance
2.4.2.3.2 KL divergence
2.4.3 Temporal classification and clustering
2.4.3.1 Temporal clustering
2.4.3.1.1 K-means clustering
2.4.3.1.2 C-means clustering
2 A3 A3 Subtractive clustering
2.4.3.1.4 Application of temporal clustering in activity recognition
2.4.3.2 Temporal classification ;
2.4.3.2.1 Decision tree
2.4.3.2.1.1 ID3 algorithm
2.4.3.2.1.2 C4.5 algorithm
2.4.3.2.1.3 Application of a decision tree in activity recognition
2.4.3.2.2 Support vector machines
2.4.3.2.2.1 Linear SVMs
2.4.3.2.2.2 Application of SVMs in activity recognition
2.4.4 Temporal-pattern discovery
2.4.4.1 Frequent-pattern discovery of activities
2.4.4.1:1 The Apriori algorithm
2.4.4.1.2 The SHIP algorithm used to make predictions in the Smart Home
2.4.4.1.3 The ALZ algorithm to make predictions in the Smart Home
2.4.4.2 Frequent episode discovery
2.4.5 Summary on temporal data mining algorithms
2.5 Discussion on qualitative vs quantitative approaches
2.6 Conclusion
3 Fuzzy temporal mining model for activities recognition 
3.1 Introduction
3.1.1 Activities’ recognition as complex mining problem
3.2 Fuzzy event mining activities
3.2.1 System viewpoint on the Smart Home
3.2.2 Fuzzy contexts of the activities
3.2.3 Fuzzy states of activities
3.2.4 Modeling of activities as a series of fuzzy events
3.2.4.1 The world
3.2.4.2 ‘ Observation
3.2.4.3 Sensor (variable) state
3.2.4.4 Fuzzy cluster centers of observations
3.2.4.5 Fuzzy sensor state
3.2.4.6 Fuzzy world state
3.2.4.7 Fuzzy activity…..
3.2.4.8 Fuzzy event
3.2.5 Activity Realization Pattern
3.2.5.1 Activity Prediction Pattern
3.2.5.2 Fuzzy context
3.3 Modeling of the activity as a fuzzy-temporal concept
3.3.1 Fuzzy temporal concept
3.3.1.1 Concept border recognition
3.3.1.2 Activity function
3.3.1.3 Simultaneous activities
3.4 Anomaly recognition and assistance provision
3.5 Conclusion
4 Validation 
4.1 Introduction
4.2 Laboratory infrastructures
4.2.1 RFID technology
4.3 Validation approach ;
4.3.1 Experimental objectives
4.3.2 Targeted applicative context
4.3.3 Chosen activities, modeling and features for simulation
4.3.3.1 Coffee-making activity
4.3.3.2 Hand-washing activity
4.3.3.3 Reading activity
4.4 Implementation and testing
4.4.1 Validation of the fuzzy event-driven modeling and mining approach
4.4.1.1 Case Study 1: fuzzy modeling of the coffee-making activity
4.4.1.2 Discussion on Case Study 1
4.4.1.3 Case Study 2: inference of ARP/APP during the coffee-making activity
4.4.2 Modeling and recognition of activities as fuzzy temporal concepts
4.4.2.1 Case Study 3: Modeling of activities as a multivariable function
4.4.2.2 Modeling of simultaneous activities
4.5 Experimental results
4.5.1 Anomaly recognition and assistance provision in the Smart Home
4.5.2 Testing the system for normal and abnormal activities
4.6 Comparison with other approaches
4.6.1 Comparison criteria
4.6.2 Comparison results
4.6.2.1 Sensory observation
4.6.2.2 Modeling
4.6.2.3 Intention recognition
4.6.2 A Recognition of correct realization
4.6.2.5 Anomaly recognition
4.7 Discussion on concept-driven recognition of activities
4.8 Conclusion concerning validation and experiments
5 General conclusion

Télécharger le rapport complet