An optimized model for mapreduce based on Hadoop | |
Zhang, Hong2,3; Wang, Xiao-Ming3; Cao, Jie2; Ma, Yan-Hong1; Guo, Yi-Rong3; Wang, Min3 | |
刊名 | Telkomnika (Telecommunication Computing Electronics and Control) |
2016-12-01 | |
卷号 | 14期号:4页码:1552-1558 |
关键词 | Data handling Dynamical systems Memory architecture Parallel architectures Distributed Fork/Join Hadoop Map-reduce Parallel |
ISSN号 | 16936930 |
DOI | 10.12928/TELKOMNIKA.v14i4.3606 |
英文摘要 | Aiming at the waste of computing resources resulting from sequential control of running mechanism of MapReduce model on Hadoop platform, Fork/Join framework has been introduced into this model to make full use of CPU resource of each node. From the perspective of fine-grained parallel data processing, combined with Fork/Join framework, a parallel and multi-thread model, this paper optimizes MapReduce model and puts forward a MapReduce+Fork/Join programming model which is a distributed and parallel architecture combined with coarse-grained and fine-grained on Hadoop platform to Support two-tier levels of parallelism architecture both in shared and distributed memory machines. A test is made under the environment of Hadoop cluster composed of four nodes. And the experimental results prove that this model really can improve performance and efficiency of the whole system and it is not only suitable for handling tasks with data intensive but also tasks with computing intensive. it is an effective optimization and improvement to the MapReduce model of big data processing. © 2016 Universitas Ahmad Dahlan. |
语种 | 英语 |
出版者 | Universitas Ahmad Dahlan |
内容类型 | 期刊论文 |
源URL | [http://ir.lut.edu.cn/handle/2XXMBERH/112994] |
专题 | 计算机与通信学院 兰州理工大学 |
作者单位 | 1.State Grid Gansu Electric Company, Lanzhou; 730030, China 2.College of Computer and Communication, Lanzhou University of Technology, Lanzhou; 730050, China; 3.College of Electrical and Information Engineering\, Lanzhou University of Technology, Lanzhou; 730050, China; |
推荐引用方式 GB/T 7714 | Zhang, Hong,Wang, Xiao-Ming,Cao, Jie,et al. An optimized model for mapreduce based on Hadoop[J]. Telkomnika (Telecommunication Computing Electronics and Control),2016,14(4):1552-1558. |
APA | Zhang, Hong,Wang, Xiao-Ming,Cao, Jie,Ma, Yan-Hong,Guo, Yi-Rong,&Wang, Min.(2016).An optimized model for mapreduce based on Hadoop.Telkomnika (Telecommunication Computing Electronics and Control),14(4),1552-1558. |
MLA | Zhang, Hong,et al."An optimized model for mapreduce based on Hadoop".Telkomnika (Telecommunication Computing Electronics and Control) 14.4(2016):1552-1558. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论