A Yarn-Based Energy-Aware Scheduling Method for Big Data Applications Under Deadline Constraints
dc.authorid | Jafari Navimipour, Nima/0000-0002-5514-5536 | |
dc.authorid | Jabbehdari, Sam/0000-0001-5168-5271 | |
dc.authorid | Rahmani, Amir Masoud/0000-0001-8641-6119 | |
dc.authorid | Shabestari, Fatemeh/0000-0003-1926-4674 | |
dc.authorwosid | Jafari Navimipour, Nima/AAF-5662-2021 | |
dc.authorwosid | Jabbehdari, Sam/AAO-8396-2021 | |
dc.authorwosid | Rahmani, Amir Masoud/K-2702-2013 | |
dc.contributor.author | Shabestari, Fatemeh | |
dc.contributor.author | Jafari Navimipour, Nima | |
dc.contributor.author | Rahmani, Amir Masoud | |
dc.contributor.author | Navimipour, Nima Jafari | |
dc.contributor.author | Jabbehdari, Sam | |
dc.date.accessioned | 2023-10-19T15:12:38Z | |
dc.date.available | 2023-10-19T15:12:38Z | |
dc.date.issued | 2022 | |
dc.department-temp | [Shabestari, Fatemeh; Rahmani, Amir Masoud] Islamic Azad Univ, Dept Comp Engn, Sci & Res Branch, Tehran, Iran; [Rahmani, Amir Masoud] Natl Yunlin Univ Sci & Technol, Future Technol Res Ctr, 123 Univ Rd,Sect 3, Touliu 64002, Yunlin, Taiwan; [Navimipour, Nima Jafari] Kadir Has Univ, Dept Comp Engn, Istanbul, Turkey; [Navimipour, Nima Jafari] Islamic Azad Univ, Dept Comp Engn, Tabriz Branch, Tabriz, Iran; [Jabbehdari, Sam] Islamic Azad Univ, Dept Comp Engn, North Tehran Branch, Tehran, Iran | en_US |
dc.description.abstract | Hadoop is a distributed framework for processing big data. One of the critical parts of Hadoop is YARN, which carries out scheduling and resource management. A scheduling algorithm should consider multiple objectives. However, YARN schedulers do not consider the Service Level Agreement (SLA) and the energy-related issues. The present paper proposes an energy-efficient deadline-aware model for the scheduling problem. The scheduling issue is an NP-hard problem regarding the deadline of applications and reducing energy. Hence, an Energy-efficient Deadline-aware Scheduling Algorithm based on the Moth-Flame Optimization algorithm (EDSA-MFO) is suggested to minimize the energy consumption and execute the application within a given soft deadline. Moreover, the earliest deadline first-based (EDF-based) heuristic approach is proposed to decode a moth into a scheduling solution. The algorithm is implemented for both static and dynamic scheduling. To evaluate the performance of the proposed algorithm, extensive simulations are conducted. The outcomes demonstrated that the suggested method could find near-optimal scheduling. It outperforms the YARN default FIFO scheduler, EDF, the energy-aware greedy algorithm (EAGA), and the deadline-aware energy-efficient MapReduce scheduling algorithm for YARN (EMRSAY) in total cluster energy consumption and meeting job deadline. | en_US |
dc.identifier.citationcount | 3 | |
dc.identifier.doi | 10.1007/s10723-022-09627-w | en_US |
dc.identifier.issn | 1570-7873 | |
dc.identifier.issn | 1572-9184 | |
dc.identifier.issue | 4 | en_US |
dc.identifier.scopus | 2-s2.0-85141147374 | en_US |
dc.identifier.scopusquality | Q1 | |
dc.identifier.uri | https://doi.org/10.1007/s10723-022-09627-w | |
dc.identifier.uri | https://hdl.handle.net/20.500.12469/5495 | |
dc.identifier.volume | 20 | en_US |
dc.identifier.wos | WOS:000879023000001 | en_US |
dc.identifier.wosquality | Q1 | |
dc.khas | 20231019-WoS | en_US |
dc.language.iso | en | en_US |
dc.publisher | Springer | en_US |
dc.relation.ispartof | Journal of Grid Computing | en_US |
dc.relation.publicationcategory | Makale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı | en_US |
dc.rights | info:eu-repo/semantics/closedAccess | en_US |
dc.scopus.citedbyCount | 5 | |
dc.subject | Virtual Machine | En_Us |
dc.subject | Optimization | En_Us |
dc.subject | Efficiency | En_Us |
dc.subject | Algorithm | En_Us |
dc.subject | Consolidation | En_Us |
dc.subject | Management | En_Us |
dc.subject | Jobs | En_Us |
dc.subject | Virtual Machine | |
dc.subject | Optimization | |
dc.subject | Hadoop | en_US |
dc.subject | Efficiency | |
dc.subject | Scheduling | en_US |
dc.subject | Algorithm | |
dc.subject | Deadline | en_US |
dc.subject | Consolidation | |
dc.subject | Energy efficiency | en_US |
dc.subject | Management | |
dc.subject | Metaheuristic | en_US |
dc.subject | Jobs | |
dc.subject | Moth-Flame | en_US |
dc.title | A Yarn-Based Energy-Aware Scheduling Method for Big Data Applications Under Deadline Constraints | en_US |
dc.type | Article | en_US |
dc.wos.citedbyCount | 4 | |
dspace.entity.type | Publication | |
relation.isAuthorOfPublication | 0fb3c7a0-c005-4e5f-a9ae-bb163df2df8e | |
relation.isAuthorOfPublication.latestForDiscovery | 0fb3c7a0-c005-4e5f-a9ae-bb163df2df8e |
Files
Original bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- 5495.pdf
- Size:
- 3.41 MB
- Format:
- Adobe Portable Document Format
- Description:
- Tam Metin / Full Text