Open Access. Powered by Scholars. Published by Universities.®

Engineering Commons™

Open Access. Powered by Scholars. Published by Universities.®

Computer Engineering

Institution
Keyword
Publication Year
Publication
Publication Type
File Type

Articles 8521 - 8550 of 25760

Full-Text Articles in Engineering

Opinion Analysis For Emotional Classification On Emoji Tweets Using The Naïve Bayes Algorithm, Siti Sendari, Ilham Ari Elbaith Zaeni, Dian Candra Lestari, Hanny Prasetya Hariyadi Jul 2020

Opinion Analysis For Emotional Classification On Emoji Tweets Using The Naïve Bayes Algorithm, Siti Sendari, Ilham Ari Elbaith Zaeni, Dian Candra Lestari, Hanny Prasetya Hariyadi

Knowledge Engineering and Data Science

Opinion Analysis is a research study needed to social media, since the content could become a trending topic and has a significant impact on social life. One of the social media that have a big contribution to cyberspace and information development is Twitter. In the Twitter application, users can insert images that represent emotions, facial expressions, or icons. Emoji is a graphic symbol in the form of an image to express a thing, with the Emoji, a text can be read and understood according to its meaning because the image represents it. Of the several things that have been mentioned …


Power-Over-Tether Uas Leveraged For Nearly-Indefinite Meteorological Data Acquisition, Daniel Rico, Carrick Detweiler, Francisco Muñoz-Arriola Jul 2020

Power-Over-Tether Uas Leveraged For Nearly-Indefinite Meteorological Data Acquisition, Daniel Rico, Carrick Detweiler, Francisco Muñoz-Arriola

School of Computing: Dissertations, Theses, and Student Research

Use of unmanned aerial systems (UASs) in agriculture has risen in the past decade. These systems are key to modernizing agriculture. UASs collect and elucidate data previously difficult to obtain and used to help increase agricultural efficiency and production. Typical commercial off-the-shelf (COTS) UASs are limited by small payloads and short flight times. Such limits inhibit their ability to provide abundant data at multiple spatiotemporal scales. In this paper, we describe the design and construction of the tethered aircraft unmanned system (TAUS), which is a novel power-over-tether UAS leveraging the physical presence of the tether to launch multiple sensors along …


Vector Magneto-Optical Generalized Ellipsometry On Magnetic Slanted Columnar Heterostructured Thin Films, Chad Briley Jul 2020

Vector Magneto-Optical Generalized Ellipsometry On Magnetic Slanted Columnar Heterostructured Thin Films, Chad Briley

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

Modern material growth techniques allow for nano-engineering highly complex three dimensionally nanostructured materials. These nano-engineered materials possess highly anisotropic physical properties that are significantly different from that of their bulk counterparts. The magnetization properties of nano-engineered materials can be modified through a close range interaction known as magnetic exchange. These materials are referred to as magnetic exchange-coupled materials. Exchange-coupled magnetic materials are composite magnetic materials where the magnetization of one material is influenced by the magnetization state of the neighboring materials.

The author describes the creation of a representative sample set of exchange-coupled nanoengineered magnetic materials. These materials are created …


Machine Learning For The Internet Of Things: Applications, Implementation, And Security, Vishalini Laguduva Ramnath Jul 2020

Machine Learning For The Internet Of Things: Applications, Implementation, And Security, Vishalini Laguduva Ramnath

USF Tampa Graduate Theses and Dissertations

Artificial intelligence and ubiquitous sensor systems have seen tremendous advances in recent times, resulting in groundbreaking impact across domains such as healthcare, entertainment, and transportation through a collective ecosystem called the Internet of Things. The advent of 5G and improved wireless networks will further accelerate the research and development of tools in deep learning, sensor systems, and computing platforms by providing improved network latency and bandwidth. While tremendous progress has been made in the Internet of Things, current work has largely focused on building robust applications that leverage the data collected through ubiquitous sensor nodes to provide actionable rules and …


Audio-Visual Caption Evaluation Metric For People Who Are Deaf And Hard Of Hearing, Akhter Al Amin Jul 2020

Audio-Visual Caption Evaluation Metric For People Who Are Deaf And Hard Of Hearing, Akhter Al Amin

Articles

While the availability of captioned television programming has increased, the quality of this captioning is not always acceptable to Deaf and Hard of Hearing (DHH) viewers, especially for live or unscripted content, broadcast from local television stations, especially in smaller markets. There is a need for formal metrics to evaluate captioning quality, to enable audits or quality assurance. Although some current caption metrics focus on comparing the textual accuracy (comparing the caption text and accurate transcription of what was spoken), there are other properties of captions that may affect quality or usability judgments. We propose to conduct experiments with DHH …


Machine Tool Communication (Mtcomm) Method And Its Applications In A Cyber-Physical Manufacturing Cloud, S M Nahian Al Sunny Jul 2020

Machine Tool Communication (Mtcomm) Method And Its Applications In A Cyber-Physical Manufacturing Cloud, S M Nahian Al Sunny

Graduate Theses and Dissertations

The integration of cyber-physical systems and cloud manufacturing has the potential to revolutionize existing manufacturing systems by enabling better accessibility, agility, and efficiency. To achieve this, it is necessary to establish a communication method of manufacturing services over the Internet to access and manage physical machines from cloud applications. Most of the existing industrial automation protocols utilize Ethernet based Local Area Network (LAN) and are not designed specifically for Internet enabled data transmission. Recently MTConnect has been gaining popularity as a standard for monitoring status of machine tools through RESTful web services and an XML based messaging structure, but it …


Earthquake Magnitude And Grid-Based Location Prediction Using Backpropagation Neural Network, Bagus Priambodo, Wayan Firdaus Mahmudy, Muh Arif Rahman Jul 2020

Earthquake Magnitude And Grid-Based Location Prediction Using Backpropagation Neural Network, Bagus Priambodo, Wayan Firdaus Mahmudy, Muh Arif Rahman

Knowledge Engineering and Data Science

Earthquakes, a type of inevitable natural disaster, is responsible for the highest average death toll per year compared to other types of a natural disaster. Even though it is inevitable, but it can be anticipated to minimize damage and casualties, such as predicting the earthquake‘s magnitude using a neural network. In this study, a backpropagation algorithm is used to train the multilayer neural network to weekly predict the average magnitude of earthquakes in grid-based locations in Indonesia. Based on the findings in this research, the neural network is able to predict the magnitude of earthquakes in grid-based locations across Indonesia …


Machine Learning Applications In Power Systems, Xinan Wang Jul 2020

Machine Learning Applications In Power Systems, Xinan Wang

Electrical Engineering Theses and Dissertations

Machine learning (ML) applications have seen tremendous adoption in power system research and applications. For instance, supervised/unsupervised learning-based load forecasting and fault detection are classic ML topics that have been well studied. Recently, reinforcement learning-based voltage control, distribution analysis, etc., are also gaining popularity. Compared to conventional mathematical methods, ML methods have the following advantages: (i). better robustness against different system configurations due to its data-driven nature; (ii). better adaption to system uncertainties; (iii). less dependent on the modeling accuracy and validity of assumptions. However, due to the unique physics of the power grid, many problems cannot be directly solved …


Leveraging Smart Contracts For Asynchronous Group Key Agreement In Internet Of Things, Victor Youdom Kemmoe, Junggab Son Jul 2020

Leveraging Smart Contracts For Asynchronous Group Key Agreement In Internet Of Things, Victor Youdom Kemmoe, Junggab Son

Master of Science in Computer Science Theses

Group Key Agreement (GKA) mechanism plays a crucial role in the realization of various secure applications in various networks such as, but not limited to, sensor networks, Internet of Things (IoT), vehicular networks, social networks, and so on. To be suitable for IoT, GKA must satisfy several critical requirements. First, a GKA mechanism must be robust against a compromised device attack and satisfy essential secrecy definitions without the existence of a Trusted Third Party (TTP). TTP is often used by IoT devices in the establishment of ad hoc networks and usually, these devices are resource-constrained. Second, the GKA mechanism must …


Rethinking The Weakness Of Stream Ciphers And Its Application To Encrypted Malware Detection, William T. Stone, Junggab Son Jul 2020

Rethinking The Weakness Of Stream Ciphers And Its Application To Encrypted Malware Detection, William T. Stone, Junggab Son

Master of Science in Computer Science Theses

Encryption key use is a critical component to the security of a stream cipher: because many implementations simply consist of a key scheduling algorithm and logical exclusive or (XOR), an attacker can completely break the cipher by XORing two ciphertexts encrypted under the same key, revealing the original plaintexts and the key itself. The research presented in this paper reinterprets this phenomenon, using repeated-key cryptanalysis for stream cipher identification. It has been found that a stream cipher executed under a fixed key generates patterns in each character of the ciphertexts it produces and that these patterns can be used to …


On The Robustness Of Bayesian Network Learning Algorithms Against Malicious Attacks, Noah Joseph Geveke Jul 2020

On The Robustness Of Bayesian Network Learning Algorithms Against Malicious Attacks, Noah Joseph Geveke

Theses and Dissertations

Bayesian networks are effective tools for discovering relationships between variables in a data set. Algorithms that learn Bayesian networks from data fall into three categories: constraint-based, score-based, and hybrid. Hybrid algorithms contain a constraint testing sub-procedure as well as a score function to create the network. Malicious changes to the training set can cause invalid networks that do not model the true data. The effects of these changes have been demonstrated using the PC algorithm, a constraint-based algorithm. In this thesis a method was developed to measure the robustness of various algorithms to determine potential malicious changes. The robustness analysis …


Identification Of Users Via Ssh Timing Attack, Thomas J. Flucke Jul 2020

Identification Of Users Via Ssh Timing Attack, Thomas J. Flucke

Master's Theses

Secure Shell, a tool to securely access and run programs on a remote machine, is an important tool for both system administrators and developers alike. The technology landscape is becoming increasingly distributed and reliant on tools such as Secure Shell to protect information as a user works on a system remotely. While Secure Shell accounts for the abuses the security of older tools such as telnet overlook, it still has fundamental vulnerabilities which leak information about both the user and their activities through timing attacks. The OpenSSH client, the implementation included in all Linux, Mac, and Windows computers, sends each …


Human Behavior Is A Significant Flaw In Maintaining Cyber Security, Elizabeth Jackson Jul 2020

Human Behavior Is A Significant Flaw In Maintaining Cyber Security, Elizabeth Jackson

Cybersecurity Undergraduate Research Showcase

Human behavior and data security utilization must be intertwined; in order to mitigate the negative effects of cyber attacks. No consumer wants their data hacked, breached, stolen, shared or wiped out. It is imperative to survey the type of education is needed to keep users safe and interested in securing their data. This can be done by simply seeking out the consumer's view of data security. The information obtained would allow the cybersecurity community to offer a simple way for consumers to protect their mobile data. There is a constant interaction between human behavior and the need for increased data …


Query Rewriting With Thesaurus-Based For Handling Semantic Heterogeneity In Database Integration, I Made Riyan Adi Nugroho, I Wayan Budi Sentana Jul 2020

Query Rewriting With Thesaurus-Based For Handling Semantic Heterogeneity In Database Integration, I Made Riyan Adi Nugroho, I Wayan Budi Sentana

Knowledge Engineering and Data Science

Nowadays, studies on handling semantic heterogeneity still become a challenge for researcher. Several methods have been used to solve these problems, one of which is query rewriting, implemented by rewriting a query into the latest one by using the selected schema. Semantic query rewriting needs a framework in order to identify the connection through the data schema sources. This line is used as a basis for scheme selection. Also, ontology is a model which often be used in these specific cases. The lack of ontology becomes a significant problem that usually seen. Therefore, this paper will describe an alternative framework …


Human Intestinal Condition Identification Based-On Blended Spatial And Morphological Feature Using Artificial Neural Network Classifier, Ummi Athiyah, Arif Wirawan Muhammad, Ahmad Azhari Jul 2020

Human Intestinal Condition Identification Based-On Blended Spatial And Morphological Feature Using Artificial Neural Network Classifier, Ummi Athiyah, Arif Wirawan Muhammad, Ahmad Azhari

Knowledge Engineering and Data Science

Colon cancer is a type of disease that attacks the intestinal walls cell of humans. Colorectal endoscopic screening technique is a common step carried out by the health expert/gynecologist to determine the condition of the human intestine. Manual interpretation requires quite a long time to reach a result. Along with the development of increasingly advanced digital computing techniques, then some of the weaknesses of the manually endoscopic image interpretation analysis model can be corrected by automating the detection process of the presence or absence of cancerous cells in the gut. Identification of human intestinal conditions using an artificial neural network …


Parallelization Of Partitioning Around Medoids (Pam) In K-Medoids Clustering On Gpu, Adhi Prahara, Dewi Pramudi Ismi, Ahmad Azhari Jul 2020

Parallelization Of Partitioning Around Medoids (Pam) In K-Medoids Clustering On Gpu, Adhi Prahara, Dewi Pramudi Ismi, Ahmad Azhari

Knowledge Engineering and Data Science

K-medoids clustering is categorized as partitional clustering. K-medoids offers better result when dealing with outliers and arbitrary distance metric also in the situation when the mean or median does not exist within data. However, k-medoids suffers a high computational complexity. Partitioning Around Medoids (PAM) has been developed to improve k-medoids clustering, consists of build and swap steps and uses the entire dataset to find the best potential medoids. Thus, PAM produces better medoids than other algorithms. This research proposes the parallelization of PAM in k-medoids clustering on GPU to reduce computational time at the swap step of PAM. The parallelization …


A Multicriteria Aware Optimal Routing Approach For Enhancing Quality Of Service In Device To Device Communication, Tilwari Valmik Jul 2020

A Multicriteria Aware Optimal Routing Approach For Enhancing Quality Of Service In Device To Device Communication, Tilwari Valmik

Student Works (2020-2029)

As the world is moving towards the digitalization era, increasing demands of higher data rates, energy effiffifficiency, and seamless connectivity are skyrocketing. Device-to-Device (D2D) communication is one of the key technologies for future Fifth Generation (5G) network. D2D communication technology enhances network coverage, boosts spectral efficiency, has low latency, and enables the devices to communicate with each other, with partial or none involvement of network infrastructure. Therefore, factors of such nature make D2D communication a promising medium guarantying reliability to several telecommunications scenarios. D2D caters to all the needs of it’s users, from the high demand of peer-to-peer users for …


Optimal Network Reconfiguration And Distributed Generation Sizing With Consideration Of Protection System, Abdul Rahim Mohamad Norshahrani Jul 2020

Optimal Network Reconfiguration And Distributed Generation Sizing With Consideration Of Protection System, Abdul Rahim Mohamad Norshahrani

Student Works (2020-2029)

Research on network reconfiguration (NR) considering distributed generation (DG) commonly focus on the issues of power loss, voltage deviation, DG sizing and its placement. These works are important and required in the planning stage of the distribution networks. In most of the past works, the protection system is neglected in this stage although it is essential to prevent the network from damages during operation. Without proper consideration of protection system settings and coordination during NR, the protection system may operate inaccurately. This happens due to changes in current flow and fault level when the network changes its configuration. As a …


Dynamic Modeling And Optimal Design For Net Zero Energy Houses Including Hybrid Electric And Thermal Energy Storage, Huangjie Gong, Vandana Rallabandi, Dan M. Ionel, Donald Colliver, Shaun Duerr, Cristinel Ababei Jul 2020

Dynamic Modeling And Optimal Design For Net Zero Energy Houses Including Hybrid Electric And Thermal Energy Storage, Huangjie Gong, Vandana Rallabandi, Dan M. Ionel, Donald Colliver, Shaun Duerr, Cristinel Ababei

Electrical and Computer Engineering Faculty Research and Publications

Net zero energy (NZE) houses purchase zero net metered electricity from the grid over a year. Technical challenges brought forth by NZE homes are related to the intermittent nature of solar generation, and are due to the fact that peak solar generation and load are not coincident. This leads to a large rate of change of load, and in case of high PV penetration communities, often requires the installation of gas power plants to service this variability. This article proposes a hybrid energy storage system including batteries and a variable power electric water heater which enables the NZE homes to …


Adaptive Large Neighborhood Search For Vehicle Routing Problem With Cross-Docking, Aldy Gunawan, Audrey Tedja Widjaja, Pieter Vansteenwegen, Vincent F. Yu Jul 2020

Adaptive Large Neighborhood Search For Vehicle Routing Problem With Cross-Docking, Aldy Gunawan, Audrey Tedja Widjaja, Pieter Vansteenwegen, Vincent F. Yu

Research Collection School Of Computing and Information Systems

Cross-docking is considered as a method to manage and control the inventory flow, which is essential in the context of supply chain management. This paper studies the integration of the vehicle routing problem with cross-docking, namely VRPCD which has been extensively studied due to its ability to reducethe overall costs occurring in a supply chain network. Given a fleet of homogeneous vehicles for delivering a single type of product from suppliers to customers through a cross-dock facility, the objective of VRPCD is to determine the number of vehicles used and the corresponding vehicle routes, such that the vehicleoperational and transportation …


Skin-Mimo: Vibration-Based Mimo Communication Over Human Skin, Dong Ma, Yuezhong Wu, Ming Ding, Mahbub Hassan, Wen Hu Jul 2020

Skin-Mimo: Vibration-Based Mimo Communication Over Human Skin, Dong Ma, Yuezhong Wu, Ming Ding, Mahbub Hassan, Wen Hu

Research Collection School Of Computing and Information Systems

We explore the feasibility of Multiple-Input-Multiple-Output (MIMO) communication through vibrations over human skin. Using off-the-shelf motors and piezo transducers as vibration transmitters and receivers, respectively, we build a 2x2 MIMO testbed to collect and analyze vibration signals from real subjects. Our analysis reveals that there exist multiple independent vibration channels between a pair of transmitter and receiver, confirming the feasibility of MIMO. Unfortunately, the slow ramping of mechanical motors and rapidly changing skin channels make it impractical for conventional channel sounding based channel state information (CSI) acquisition, which is critical for achieving MIMO capacity gains. To solve this problem, we …


Relational Joins On Gpus For In-Memory Database Query Processing, Ran Rui Jun 2020

Relational Joins On Gpus For In-Memory Database Query Processing, Ran Rui

USF Tampa Graduate Theses and Dissertations

Relational join processing is one of the core functionalities in database management systems. Implementing join algorithms on parallel platforms, especially modern GPUs, has gain a lot of momentum in the past decade. This dissertation addresses the following issues on GPU join algorithms. First, we present empirical evaluations of a state-of-the-art work on GPU-based join processing. Since 2008, the compute capabilities of GPUs have increased following a pace faster than that of the multi-core CPUs. We run a comprehensive set of experiments to study how join operations can benefit from such rapid expansion of GPU capabilities. We also present improved GPU …


Moving Targets: Addressing Concept Drift In Supervised Models For Hacker Communication Detection, Susan Mckeever, Brian Keegan, Andrei Quieroz Jun 2020

Moving Targets: Addressing Concept Drift In Supervised Models For Hacker Communication Detection, Susan Mckeever, Brian Keegan, Andrei Quieroz

Conference papers

Abstract—In this paper, we are investigating the presence of concept drift in machine learning models for detection of hacker communications posted in social media and hacker forums. The supervised models in this experiment are analysed in terms of performance over time by different sources of data (Surface web and Deep web). Additionally, to simulate real-world situations, these models are evaluated using time-stamped messages from our datasets, posted over time on social media platforms. We have found that models applied to hacker forums (deep web) presents an accuracy deterioration in less than a 1-year period, whereas models applied to Twitter (surface …


Simulation-Based An Exploratory Analysis Of Drone Air Strike On Saudi Oilfields, Jiajun Zhang, Xiaochao Qian, Jinpeng Zhou, Xuanling Wang, Zhifeng Lu Jun 2020

Simulation-Based An Exploratory Analysis Of Drone Air Strike On Saudi Oilfields, Jiajun Zhang, Xiaochao Qian, Jinpeng Zhou, Xuanling Wang, Zhifeng Lu

Journal of System Simulation

Abstract: The incident of air strike in Sandi Arabia in September 2019 is taken as an example,A rapid analysis method for crisis events based on simulation is proposed. Combining the simulation with the exploratory analysis, the uncertainty of crisis events is comprehensively analyzed and dealt with. According to the public information of the incidents details from Sandi Arabia, Yemen, the combat situation of the air strike is replayed based on the wargaming. The rationality and authenticity of the details in the event statement is verified and can also provide reference to the development and combat research of the “Low, Small …


Multi-Tank Adaptive Scheduling Algorithm And Simulation For Ship Stability Realization, Xiong Yong, Xuanzhuo Liang, Zhang Jia Jun 2020

Multi-Tank Adaptive Scheduling Algorithm And Simulation For Ship Stability Realization, Xiong Yong, Xuanzhuo Liang, Zhang Jia

Journal of System Simulation

Abstract: Aiming at the influence of the oil tank on the floating state and stability of the ship, the multi-tank adaptive dispatching control algorithm to maintain the stability of the ship is studied. The automatic floating state adjustment simulation system of the ship is realized by the WinCC (Windows Control Center) configuration software. For the system architecture, the Boolean logic table is used to construct the connection relationships of the oil tanks, pumps and pipelines, and the amount of oil exchange between the cabins is calculated according to the floating equation, and the balance of the floating state in the …


Simulation-Based Effectiveness Evaluation For System-Of-System Under Multi-Equipment Collaboration, Ma Ning, Shenglin Lin Jun 2020

Simulation-Based Effectiveness Evaluation For System-Of-System Under Multi-Equipment Collaboration, Ma Ning, Shenglin Lin

Journal of System Simulation

Abstract: Complex coordination exists among the various equipment in the system-of-system (SoS) operational process. The influence relationship among the various indexes is ignored in the traditional effectiveness evaluation methods. Aiming at this, a new effectiveness evaluation approach for the SoS is proposed. Numerous evaluation data is analyzed, based on the association rules mining, to obtain the influence among the effectiveness indexes. The networked evaluation index system is constructed, and the effectiveness integration evaluation is realized based on the analytic network process. The feasibility and validity of the proposed method are verified by evaluating the effectiveness of an air defense SoS.


Identification Method For Fractional-Order Systems Based On Haar Wavelet Operational Matrix, Yuanlu Li, Li Teng, Baoying Liu Jun 2020

Identification Method For Fractional-Order Systems Based On Haar Wavelet Operational Matrix, Yuanlu Li, Li Teng, Baoying Liu

Journal of System Simulation

Abstract: Studies have shown that the dynamical systems can be more accurately described by the fractional-order systems because its order can be any number. That’s why the fractional-order systems are being paid more and more attention. However, how to create a fractional-order system is still in the exploratory stage. Considering the nonlocal features of the fractional differentiation, a method for the fractional-order system identification is proposed by taking the Haar wavelet operational matrix. The proposed method can reduce the dimension of the operational matrix by abandoning the high frequency coefficients of the input and output signals so that the buffer …


Classification Of Chest X-Ray Disease Based On Convolutional Neural Network, Huang Xin, Fang Yu, Mengdan Gu Jun 2020

Classification Of Chest X-Ray Disease Based On Convolutional Neural Network, Huang Xin, Fang Yu, Mengdan Gu

Journal of System Simulation

Abstract: The artificial intelligence technology can effectively assist the chest X-ray diagnosis. On the basis of the analysis of Chinese reports of chest X-rays, a labeling method of the thoracic disease classification for the chest abnormal parts is proposed and a dataset of the thoracic disease classification labels is complied. The thoracic disease classification is evaluated through four kinds of convolutional neural networks, AlexNet, VGGNet, ResNet and DenseNet and through three kinds of training methods, direct training, ImageNet pre-training and Chest X-14 pre-training. The result shows that the more complicated convolutional neural network with the more parameters, the better performance …


Managing Off-Grid Power Use For Solar Fueled Residences With Smart Appliances, Prices-To-Devices And Iot, Donnelle L. January Jun 2020

Managing Off-Grid Power Use For Solar Fueled Residences With Smart Appliances, Prices-To-Devices And Iot, Donnelle L. January

USF Tampa Graduate Theses and Dissertations

Electricity availability has a profound impact in day to day life. Activities such as lighting homes, charging cellphones, and running appliances all depend on electricity use. Around 1 billion people around the world do not have access to electricity, and the majority of these people hail from developing countries in remote areas. Moreover, many of these remote areas lack utility grid access due to the infeasible cost of extending the grid to remote communities. To combat this, various projects on microgrids have been implemented in order to distribute power to these off-grid areas. Nanogrids are defined as “a single domain …


Study Of Parallelized Graph Clustering Algorithm Based On Spark, Dongjiang Liu, Jianhui Li Jun 2020

Study Of Parallelized Graph Clustering Algorithm Based On Spark, Dongjiang Liu, Jianhui Li

Journal of System Simulation

Abstract: The parallelized graph clustering algorithm is researched. A new parallelized graph clustering algorithm is proposed based on Spark. As the top operation of Spark occupies a lot of memory space, a new algorithm which is used to substitute the top operation is proposed to reduce the memory consumption. By improving bottom up hierarchical clustering algorithm, the speed of the proposed algorithm is improved. A new data filtering method based on the feature of graph data is proposed. By the method, the running time and memory space comsuption is reduced greatly. The reason of the high efficiency of this filtering …