Open Access. Powered by Scholars. Published by Universities.®

OS and Networks Commons

Open Access. Powered by Scholars. Published by Universities.®

Computer Engineering

Institution
Keyword
Publication Year
Publication
Publication Type

Articles 61 - 90 of 197

Full-Text Articles in OS and Networks

Scaling Human Activity Recognition Via Deep Learning-Based Domain Adaptation, Md Abdullah Hafiz Khan, Nirmalya Roy, Archan Misra Mar 2018

Scaling Human Activity Recognition Via Deep Learning-Based Domain Adaptation, Md Abdullah Hafiz Khan, Nirmalya Roy, Archan Misra

Research Collection School Of Computing and Information Systems

We investigate the problem of making human activityrecognition (AR) scalable–i.e., allowing AR classifiers trainedin one context to be readily adapted to a different contextualdomain. This is important because AR technologies can achievehigh accuracy if the classifiers are trained for a specific individualor device, but show significant degradation when the sameclassifier is applied context–e.g., to a different device located ata different on-body position. To allow such adaptation withoutrequiring the onerous step of collecting large volumes of labeledtraining data in the target domain, we proposed a transductivetransfer learning model that is specifically tuned to the propertiesof convolutional neural networks (CNNs). Our model, …


Collaborative Fall Detection Using Smartphone And Kinect, Xue Li, Lanshun Nie, Hanchuan Xu, Xianzhi Wang Jan 2018

Collaborative Fall Detection Using Smartphone And Kinect, Xue Li, Lanshun Nie, Hanchuan Xu, Xianzhi Wang

Research Collection School Of Computing and Information Systems

Humanfall detection has attracted broad attentions as sensors and mobile devices are increasingly adopted in real-life scenarios such as smart homes. The complexity of activities in home environments pose severe challenges to the fall detection research with respect to the detection accuracy. We propose a collaborative detection platform that combines two subsystems: a threshold-based fall detection subsystem using mobile phones and a support vector machine (SVM)-based fall detection subsystem using Kinects. Both subsystems have their respective confidence models and the platform detects falls by fusing the data of both subsystems using two methods: the logical rules-based and D-S evidence fusion …


Bikemate: Bike Riding Behavior Monitoring With Smartphones, Weixi Gu, Zimu Zhou, Yuxun Zhou, Han Zou, Yunxin Liu, Costas J. Spanos, Lin Zhang Nov 2017

Bikemate: Bike Riding Behavior Monitoring With Smartphones, Weixi Gu, Zimu Zhou, Yuxun Zhou, Han Zou, Yunxin Liu, Costas J. Spanos, Lin Zhang

Research Collection School Of Computing and Information Systems

Detecting dangerous riding behaviors is of great importance to improve bicycling safety. Existing bike safety precautionary measures rely on dedicated infrastructures that incur high installation costs. In this work, we propose BikeMate, a ubiquitous bicycling behavior monitoring system with smartphones. BikeMate invokes smartphone sensors to infer dangerous riding behaviors including lane weaving, standing pedalling and wrong-way riding. For easy adoption, BikeMate leverages transfer learning to reduce the overhead of training models for different users, and applies crowdsourcing to infer legal riding directions without prior knowledge. Experiments with 12 participants show that BikeMate achieves an overall accuracy of 86.8% for lane …


Understanding Inactive Yet Available Assignees In Github, Jing Jiang, David Lo, Xinyu Ma, Fuli Feng, Li Zhang Nov 2017

Understanding Inactive Yet Available Assignees In Github, Jing Jiang, David Lo, Xinyu Ma, Fuli Feng, Li Zhang

Research Collection School Of Computing and Information Systems

Context In GitHub, an issue or a pull request can be assigned to a specific assignee who is responsible for working on this issue or pull request. Due to the principle of voluntary participation, available assignees may remain inactive in projects. If assignees ever participate in projects, they are active assignees; otherwise, they are inactive yet available assignees (inactive assignees for short). Objective Our objective in this paper is to provide a comprehensive analysis of inactive yet available assignees in GitHub. Method We collect 2,374,474 records of activities in 37 popular projects, and 797,756 records of activities in 687 projects …


Intent Recognition In Smart Living Through Deep Recurrent Neural Networks, Xiang Zhang, Lina Yao, Chaoran Huang, Quan Z. Sheng, Xianzhi Wang Nov 2017

Intent Recognition In Smart Living Through Deep Recurrent Neural Networks, Xiang Zhang, Lina Yao, Chaoran Huang, Quan Z. Sheng, Xianzhi Wang

Research Collection School Of Computing and Information Systems

Electroencephalography (EEG) signal based intent recognition has recently attracted much attention in both academia and industries, due to helping the elderly or motor-disabled people controlling smart devices to communicate with outer world. However, the utilization of EEG signals is challenged by low accuracy, arduous and time-consuming feature extraction. This paper proposes a 7-layer deep learning model to classify raw EEG signals with the aim of recognizing subjects’ intents, to avoid the time consumed in pre-processing and feature extraction. The hyper-parameters are selected by an Orthogonal Array experiment method for efficiency. Our model is applied to an open EEG dataset provided …


Improving Hpc Communication Library Performance On Modern Architectures, Matthew G. F. Dosanjh Oct 2017

Improving Hpc Communication Library Performance On Modern Architectures, Matthew G. F. Dosanjh

Computer Science ETDs

As high-performance computing (HPC) systems advance towards exascale (10^18 operations per second), they must leverage increasing levels of parallelism to achieve their performance goals. In addition to increased parallelism, machines of that scale will have strict power limitations placed on them. One direction currently being explored to alleviate those issues are many-core processors such as Intel’s Xeon Phi line. Many-core processors sacrifice clock speed and core complexity, such as out of order pipelining, to increase the number of cores on a die. While this increases floating point throughput, it can reduce the performance of serialized, synchronized, and latency sensitive code …


Ancr—An Adaptive Network Coding Routing Scheme For Wsns With Different-Success-Rate Links †, Xiang Ji, Anwen Wang, Chunyu Li, Chun Ma, Yao Peng, Dajin Wang, Qingyi Hua, Feng Chen, Dingyi Fang Aug 2017

Ancr—An Adaptive Network Coding Routing Scheme For Wsns With Different-Success-Rate Links †, Xiang Ji, Anwen Wang, Chunyu Li, Chun Ma, Yao Peng, Dajin Wang, Qingyi Hua, Feng Chen, Dingyi Fang

Department of Computer Science Faculty Scholarship and Creative Works

As the underlying infrastructure of the Internet of Things (IoT), wireless sensor networks (WSNs) have been widely used in many applications. Network coding is a technique in WSNs to combine multiple channels of data in one transmission, wherever possible, to save node’s energy as well as increase the network throughput. So far most works on network coding are based on two assumptions to determine coding opportunities: (1) All the links in the network have the same transmission success rate; (2) Each link is bidirectional, and has the same transmission success rate on both ways. However, these assumptions may not be …


Operating System Identification By Ipv6 Communication Using Machine Learning Ensembles, Adrian Ordorica Aug 2017

Operating System Identification By Ipv6 Communication Using Machine Learning Ensembles, Adrian Ordorica

Graduate Theses and Dissertations

Operating system (OS) identification tools, sometimes called fingerprinting tools, are essential for the reconnaissance phase of penetration testing. While OS identification is traditionally performed by passive or active tools that use fingerprint databases, very little work has focused on using machine learning techniques. Moreover, significantly more work has focused on IPv4 than IPv6. We introduce a collaborative neural network ensemble that uses a unique voting system and a random forest ensemble to deliver accurate predictions. This approach uses IPv6 features as well as packet metadata features for OS identification. Our experiment shows that our approach is valid and we achieve …


Multiple Attributes Decision Fusion For Wireless Sensor Networks Based On Intuitionistic Fuzzy Set, Zhenjiang Zhang, Ziqi Hao, Sherali Zeadally, Jing Zhang, Bowen Han, Han-Chieh Chao Jul 2017

Multiple Attributes Decision Fusion For Wireless Sensor Networks Based On Intuitionistic Fuzzy Set, Zhenjiang Zhang, Ziqi Hao, Sherali Zeadally, Jing Zhang, Bowen Han, Han-Chieh Chao

Information Science Faculty Publications

Decision fusion is an important issue in wireless sensor networks (WSN), and intuitionistic fuzzy set (IFS) is a novel method for dealing with uncertain data. We propose a multi-attribute decision fusion model based on IFS, which includes two aspects: data distribution-based IFS construction algorithm (DDBIFCA) and the category similarity weight-based TOPSIS intuitionistic fuzzy decision algorithm (CSWBT-IFS). The DDBIFCA is an IFS construction algorithm that transforms the original attribute values into intuitionistic fuzzy measures, and the CSWBT-IFS is an intuitionistic fuzzy aggregation algorithm improved by the traditional TOPSIS algorithm, which combines intuitionistic fuzzy values of different attributes and obtains a final …


How Technology Is Reshaping Financial Services: Essays On Consumer Behavior In Card, Channel And Cryptocurrency Services, Dan Geng Jul 2017

How Technology Is Reshaping Financial Services: Essays On Consumer Behavior In Card, Channel And Cryptocurrency Services, Dan Geng

Dissertations and Theses Collection

The financial services sector has seen dramatic technological innovations in the last several years associated with the “fintech revolution.” Major changes have taken place in channel management, credit card rewards marketing, cryptocurren-cy, and wealth management, and have influenced consumers’ banking behavior in different ways. As a consequence, there has been a growing demand for banks to rethink their business models and operations to adapt to changing consumer be-havior and counter the competitive pressure from other banks and non-bank play-ers. In this dissertation, I study consumer behavior related to different aspects of financial innovation by addressing research questions that are motivated …


Understanding Deleted File Decay On Removable Media Using Differential Analysis, James H. Jones Jr, Anurag Srivastava, Josh Mosier, Connor Anderson, Seth Buenafe May 2017

Understanding Deleted File Decay On Removable Media Using Differential Analysis, James H. Jones Jr, Anurag Srivastava, Josh Mosier, Connor Anderson, Seth Buenafe

Annual ADFSL Conference on Digital Forensics, Security and Law

Digital content created by picture recording devices is often stored internally on the source device, on either embedded or removable media. Such storage media is typically limited in capacity and meant primarily for interim storage of the most recent image files, and these devices are frequently configured to delete older files as necessary to make room for new files. When investigations involve such devices and media, it is sometimes these older deleted files that would be of interest. It is an established fact that deleted file content may persist in part or in its entirety after deletion, and identifying the …


Harnessing Predictive Models For Assisting Network Forensic Investigations Of Dns Tunnels, Irvin Homem, Panagiotis Papapetrou May 2017

Harnessing Predictive Models For Assisting Network Forensic Investigations Of Dns Tunnels, Irvin Homem, Panagiotis Papapetrou

Annual ADFSL Conference on Digital Forensics, Security and Law

In recent times, DNS tunneling techniques have been used for malicious purposes, however network security mechanisms struggle to detect them. Network forensic analysis has been proven effective, but is slow and effort intensive as Network Forensics Analysis Tools struggle to deal with undocumented or new network tunneling techniques. In this paper, we present a machine learning approach, based on feature subsets of network traffic evidence, to aid forensic analysis through automating the inference of protocols carried within DNS tunneling techniques. We explore four network protocols, namely, HTTP, HTTPS, FTP, and POP3. Three features are extracted from the DNS tunneled traffic: …


An Accidental Discovery Of Iot Botnets And A Method For Investigating Them With A Custom Lua Dissector, Max Gannon, Gary Warner, Arsh Arora May 2017

An Accidental Discovery Of Iot Botnets And A Method For Investigating Them With A Custom Lua Dissector, Max Gannon, Gary Warner, Arsh Arora

Annual ADFSL Conference on Digital Forensics, Security and Law

This paper presents a case study that occurred while observing peer-to-peer network communications on a botnet monitoring station and shares how tools were developed to discover what ultimately was identified as Mirai and many related IoT DDOS Botnets. The paper explains how researchers developed a customized protocol dissector in Wireshark using the Lua coding language, and how this enabled them to quickly identify new DDOS variants over a five month period of study.


Lightweight Data Aggregation Scheme Against Internal Attackers In Smart Grid Using Elliptic Curve Cryptography, Debiao He, Sherali Zeadally, Huaqun Wang, Qin Liu May 2017

Lightweight Data Aggregation Scheme Against Internal Attackers In Smart Grid Using Elliptic Curve Cryptography, Debiao He, Sherali Zeadally, Huaqun Wang, Qin Liu

Information Science Faculty Publications

Recent advances of Internet and microelectronics technologies have led to the concept of smart grid which has been a widespread concern for industry, governments, and academia. The openness of communications in the smart grid environment makes the system vulnerable to different types of attacks. The implementation of secure communication and the protection of consumers’ privacy have become challenging issues. The data aggregation scheme is an important technique for preserving consumers’ privacy because it can stop the leakage of a specific consumer’s data. To satisfy the security requirements of practical applications, a lot of data aggregation schemes were presented over the …


Network-On-Chip Based H.264 Video Decoder On A Field Programmable Gate Array, Ian Barge Apr 2017

Network-On-Chip Based H.264 Video Decoder On A Field Programmable Gate Array, Ian Barge

Master's Theses (2009 -)

This thesis develops the first fully network-on-chip (NoC) based h.264 video decoder implemented in real hardware on a field programmable gate array (FPGA). This thesis starts with an overview of the h.264 video coding standard and an introduction to the NoC communication paradigm. Following this, a series of processing elements (PEs) are developed which implement the component algorithms making up the h.264 video decoder. These PEs, described primarily in VHDL with some Verilog and C, are then mapped to an NoC which is generated using the CONNECT NoC generation tool. To demonstrate the scalability of the proposed NoC based design, …


Smart Underground Antenna Arrays: A Soil Moisture Adaptive Beamforming Approach, Abdul Salam, Mehmet C. Vuran Jan 2017

Smart Underground Antenna Arrays: A Soil Moisture Adaptive Beamforming Approach, Abdul Salam, Mehmet C. Vuran

School of Computing: Technical Reports

In this paper, a novel framework for underground beamforming using adaptive antenna arrays is presented. Based on the analysis of propagation in wireless underground channel, a theoretical model is developed which uses soil moisture information and feedback mechanism to improve performance wireless underground communications. Array element in soil has been analyzed empirically and impacts of soil type and soil moisture on return loss and resonant frequency are investigated. Beam patterns are investigated to communicate with both underground and above ground devices. Depending on the incident angle, refraction from soil-air interface has the adverse effects in the UG communications. It is …


Android Drone: Remote Quadcopter Control With A Phone, Aubrey John Russell Dec 2016

Android Drone: Remote Quadcopter Control With A Phone, Aubrey John Russell

Computer Engineering

The purpose of the “Android Drone” project was to create a quadcopter that can be controlled by user input sent over the phone’s Wi-Fi connection or 4G internet connection. Furthermore, the purpose was also to be able to receive live video feedback over the internet connection, thus making the drone an inexpensive option compared to other, equivalent drones that might cost thousands of dollars. Not only that, but the Android phone also has a host of other useful features that could be utilized by the drone: this includes GPS, pathing, picture taking, data storage, networking and TCP/IP, a Java software …


Summarization Of Egocentric Videos: A Comprehensive Survey, Ana Garcia Del Molino, Cheston Tan, Joo-Hwee Lim, Ah-Hwee Tan Nov 2016

Summarization Of Egocentric Videos: A Comprehensive Survey, Ana Garcia Del Molino, Cheston Tan, Joo-Hwee Lim, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

The introduction of wearable video cameras (e.g., GoPro) in the consumer market has promoted video life-logging, motivating users to generate large amounts of video data. This increasing flow of first-person video has led to a growing need for automatic video summarization adapted to the characteristics and applications of egocentric video. With this paper, we provide the first comprehensive survey of the techniques used specifically to summarize egocentric videos. We present a framework for first-person view summarization and compare the segmentation methods and selection algorithms used by the related work in the literature. Next, we describe the existing egocentric video datasets …


Passively Testing Routing Protocols In Wireless Sensor Networks, Xiaoping Che, Stephane Maag, Hwee-Xian Tan, Hwee-Pink Tan Jul 2016

Passively Testing Routing Protocols In Wireless Sensor Networks, Xiaoping Che, Stephane Maag, Hwee-Xian Tan, Hwee-Pink Tan

Research Collection School Of Computing and Information Systems

Smart systems are today increasingly developed with the number of wireless sensor devices that drastically increases. They are implemented within several contexts through our environment. Thus, sensed data transported in ubiquitous systems are important and the way to carry them must be efficient and reliable. For that purpose, several routing protocols have been proposed to wireless sensor networks (WSN). However, one stage that is often neglected before their deployment, is the conformance testing process, a crucial and challenging step. Active testing techniques commonly used in wired networks are not suitable to WSN and passive approaches are needed. While some works …


Adaptive Scaling Of Cluster Boundaries For Large-Scale Social Media Data Clustering, Lei Meng, Ah-Hwee Tan, Donald C. Wunsch Dec 2015

Adaptive Scaling Of Cluster Boundaries For Large-Scale Social Media Data Clustering, Lei Meng, Ah-Hwee Tan, Donald C. Wunsch

Research Collection School Of Computing and Information Systems

The large scale and complex nature of social media data raises the need to scale clustering techniques to big data and make them capable of automatically identifying data clusters with few empirical settings. In this paper, we present our investigation and three algorithms based on the fuzzy adaptive resonance theory (Fuzzy ART) that have linear computational complexity, use a single parameter, i.e., the vigilance parameter to identify data clusters, and are robust to modest parameter settings. The contribution of this paper lies in two aspects. First, we theoretically demonstrate how complement coding, commonly known as a normalization method, changes the …


Shopminer: Mining Customer Shopping Behavior In Physical Clothing Stores With Passive Rfids, Longfei Shangguan, Zimu Zhou, Xiaolong Zheng, Lei Yang, Yunhao Liu, Jinsong Han Nov 2015

Shopminer: Mining Customer Shopping Behavior In Physical Clothing Stores With Passive Rfids, Longfei Shangguan, Zimu Zhou, Xiaolong Zheng, Lei Yang, Yunhao Liu, Jinsong Han

Research Collection School Of Computing and Information Systems

Shopping behavior data are of great importance to understand the effectiveness of marketing and merchandising efforts. Online clothing stores are capable capturing customer shopping behavior by analyzing the click stream and customer shopping carts. Retailers with physical clothing stores, however, still lack effective methods to identify comprehensive shopping behaviors. In this paper, we show that backscatter signals of passive RFID tags can be exploited to detect and record how customers browse stores, which items of clothes they pay attention to, and which items of clothes they usually match with. The intuition is that the phase readings of tags attached on …


Self-Organizing Neural Networks Integrating Domain Knowledge And Reinforcement Learning, Teck-Hou Teng, Ah-Hwee Tan, Jacek M. Zurada May 2015

Self-Organizing Neural Networks Integrating Domain Knowledge And Reinforcement Learning, Teck-Hou Teng, Ah-Hwee Tan, Jacek M. Zurada

Research Collection School Of Computing and Information Systems

The use of domain knowledge in learning systems is expected to improve learning efficiency and reduce model complexity. However, due to the incompatibility with knowledge structure of the learning systems and real-time exploratory nature of reinforcement learning (RL), domain knowledge cannot be inserted directly. In this paper, we show how self-organizing neural networks designed for online and incremental adaptation can integrate domain knowledge and RL. Specifically, symbol-based domain knowledge is translated into numeric patterns before inserting into the self-organizing neural networks. To ensure effective use of domain knowledge, we present an analysis of how the inserted knowledge is used by …


Secure And Reliable Routing Protocol For Transmission Data In Wireless Sensor Mesh Networks, Nooh Adel Bany Muhammad May 2015

Secure And Reliable Routing Protocol For Transmission Data In Wireless Sensor Mesh Networks, Nooh Adel Bany Muhammad

Dissertations

Abstract

Sensor nodes collect data from the physical world then exchange it until it reaches the intended destination. This information can be sensitive, such as battlefield surveillance. Therefore, providing secure and continuous data transmissions among sensor nodes in wireless network environments is crucial. Wireless sensor networks (WSN) have limited resources, limited computation capabilities, and the exchange of data through the air and deployment in accessible areas makes the energy, security, and routing major concerns in WSN. In this research we are looking at security issues for the above reasons. WSN is susceptible to malicious activities such as hacking and physical …


Measuring Centralities For Transportation Networks Beyond Structures, Yew-Yih Cheng, Lee Ka Wei, Roy, Ee-Peng Lim, Feida Zhu May 2015

Measuring Centralities For Transportation Networks Beyond Structures, Yew-Yih Cheng, Lee Ka Wei, Roy, Ee-Peng Lim, Feida Zhu

Research Collection School Of Computing and Information Systems

In an urban city, its transportation network supports efficient flow of people between different parts of the city. Failures in the network can cause major disruptions to commuter and business activities which can result in both significant economic and time losses. In this paper, we investigate the use of centrality measures to determine critical nodes in a transportation network so as to improve the design of the network as well as to devise plans for coping with the network failures. Most centrality measures in social network analysis research unfortunately consider only topological structure of the network and are oblivious of …


Fresh Bytes - Connected Hydroponics For Small-Scale Growing, Jack Bowen Dec 2014

Fresh Bytes - Connected Hydroponics For Small-Scale Growing, Jack Bowen

Liberal Arts and Engineering Studies

Many users are now transitioning to small-scale hydroponics and aquaponics at home. There can be a barrier to entry with these systems as there is a delicate balance of chemicals, pH, etc. that must be maintained. There are sensors for these various components but they are either aimed at commercial production or are un-automated. Fresh Bytes is a microcomputer with sensors to detect all of these unseen components in a hydroponic system. A prototype of this microcomputer is produced along with CAD plans for more professional versions of it. The sensors are verified and future development is contemplated.


Is The Smartphone Smart In Kathmandu?, Seth Bird Dec 2014

Is The Smartphone Smart In Kathmandu?, Seth Bird

Independent Study Project (ISP) Collection

This is the extensive study of the smartphone in the developing country of Nepal, specifically the Kathmandu valley. Throughout my research I conducted various interviews with businesses, Tibetan refugees, and Nepali millennials (18yrs-33yrs) with the goal of identifying how the smartphone is used and understood. I chose the Kathmandu valley as my main area of research because the usage of smartphones in rural Nepal is extremely limited, and the valley represents the economic hub where progressive thinking flourishes. As a main objective I sought to understand how, if at all, the smartphone is used differently between Nepal and America. All …


Power Management In The Cluster System, Leping Wang Jun 2014

Power Management In The Cluster System, Leping Wang

School of Computing: Dissertations, Theses, and Student Research

With growing cost of electricity, the power management of server clusters has become an important problem. However, most previous researchers have only addressed the challenge in traditional homogeneous environments. Considering the increasing popularity of heterogeneous and virtualized systems, this thesis develops a series of efficient algorithms respectively for power management of heterogeneous soft real-time clusters and a virtualized cluster system. It is built on simple but effective mathematical models. When deployed to a new platform, the software incurs low configuration cost because no extensive performance measurements and profiling are required. Built upon optimization, queuing theory and control theory techniques, our …


Opportunistic Service Differentiation And Cloud Resource Management In Support Of Enhanced Vehicular Applications, Mohammad Ali Salahuddin Jun 2014

Opportunistic Service Differentiation And Cloud Resource Management In Support Of Enhanced Vehicular Applications, Mohammad Ali Salahuddin

Dissertations

An integral part of Intelligent Transportation Systems (ITS) are Vehicular Ad hoc Networks (VANETs), which consist of vehicles with on-board units (OBUs) and fixed road-side units (RSUs). Wireless Access in Vehicular Environment (WAVE) offers QoS via service differentiation by using application defined priorities. However, WAVE has unbounded delay and is oblivious to network load and severity of vehicles with respect to their environment. Our context severity metric innovatively enhances WAVE to be sensitive to vehicle and environment interactions. Our novel Opportunistic Service Differentiation (OSD) technique, dynamically readjusts the WAVE packet priorities to improve utilization of lower latency queues, prioritizing packets …


Conception, Design And Construction Of A Remote Wifi Vehicle Using Arduino, Clayton Broman Mar 2014

Conception, Design And Construction Of A Remote Wifi Vehicle Using Arduino, Clayton Broman

Physics

The scope of this senior project was to make a wireless vehicle controlled via Internet Protocol. This vehicle operates remotely and without direct line of sight. Commands are sent from a program running on a laptop and transmitted using a wireless router. Visual data is retrieved from a network camera, mounted on the vehicle, in real-time, to see where you are going.


Providing Flexible File-Level Data Filtering For Big Data Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson Jan 2014

Providing Flexible File-Level Data Filtering For Big Data Analytics, Lei Xu, Ziling Huang, Hong Jiang, Lei Tian, David Swanson

School of Computing: Technical Reports

The enormous amount of big data datasets impose the needs for effective data filtering technique to accelerate the analytics process. We propose a Versatile Searchable File System, VSFS, which provides a transparent, flexible and near real-time file-level data filtering service by searching files directly through the file system. Therefore, big data analytics applications can transparently utilize this filtering service without application modifications. A versatile index scheme is designed to adapt to the exploratory and ad-hoc nature of the big data analytics activities. Moreover, VSFS uses a RAM-based distributed architecture to perform file indexing. The evaluations driven by three real-world analytics …