Challenges and Security in Big Data Analysis †MyAssignmenthelp.com

Question: Discussn about the Challenges and Security in Big Data Analysis. Answer: Introduction: Big data is the large volume of data stored on the internet. The organizations are facing problems in managing the complex data for retrieval. The management of big data is capable of resolving the issues related with retrieval of information. The advancement in big data management plays a vital role in taking efficient decision related with the critical region of enterprise by providing instant information (Sinanc, 2015). The environment of big data can be characterised by handling data in petabytes, distribution and management of redundant data for storage, leveraging the processing related with parallel tasks, increasing the capabilities of data processing, efficient in insertion and retrieval of information, and central management of data. In this paper we are going to focus on the security and privacy concerns associated with the management of big data over the network and what are the possible solutions to handle the big data (Wayne, 2013). The architecture of the big data envi ronment is characterised with the 5 Vs model which are composed of variety, volume, veracity, velocity and value. The following diagram shows the characteristics of the big data environment. Requirement of security in the management of big data: The security issues associated with the big data should be handled adequately because the inefficiency of the security procedures will lead to the consequences of legal issues (Muoro, 2013). The traditional security procedures are not adequate to handle the big data available on the cloud. The management of security problems associated with the handling of big data involves the inclusion of encryption policies, detection by using honey pot, and logging technique. The increase in the volume of data available on the network will results into the birth of security and privacy challenges associated with the streamlining of the data (Munaye, 2013). The following table shows the list of security and privacy issues associated with Big data: Security and privacy issues Description Security associated with the big data infrastructure Security issues related with the development of framework for distributed programming The data stored in the non-relational databases Parallel programming infrastructure involves the large amount of data for performing its tasks simultaneously (Schmitt, 2013). It makes use of map reduction techniques for handling data. The inclusion of iterative procedures can create the scenario of privacy and security issues (Moreno, 2016). Privacy issues with non-relational data management Scalability in using the technology of data analytics and mining Security issues associated with the central management of the data Access control associated with the data management The NOSQL databases are weak in security infrastructure Data storage at central database Storing the data on transactional logs Auditing of the granular data Provenance associated with the management of central data The use of multi-tiered devices for storing the data can create the problem of accuracy and exponential increase of data. Integrity of data management Validation and filtering of end point data Monitoring of the security issues associated with the real time management system The multiple sources used for collecting the data requires special attention of security The checking of validity of the data is the major security concern (Parms, 2013) Exposure to the common security vulnerabilities No reporting of vulnerabilities associated with the internal working of the enterprise Security issues associated with the web interface The exploitation of the interface due to scripting of the cross-site architecture (Neves, 2016). Forgery associated with the injection of SQL statements The security concern should be taken under consideration for monitoring the leakages of the sensitive information. Security issues associated with the authorization techniques The password policy are not followed for the development of strong password Security services provided by the network are inefficient The attacker can hack the big data from the network due to the inefficiency of the security services. Inefficiency in the encryption policies The eavesdropping attack get associated with the IoT devices Insecurity in the interface used by the cloud environment The inefficiency in the authentication protocols can result into hacking attack Insecurity in the interface used by the mobile devices The inefficiency in the authentication protocols can result into hacking attack Inefficiency in the security configuration The configuration is of poor quality for controlling the transfer of data Physical security is not accurate The planning of the physical infrastructure is not adequate Monitoring of the real time data management The focus should be given on the infrastructure for efficient working in the real time environment. The focus should be given on the data analytics and mining techniques used for providing data in the big data environment The implementation of alert management system with every node is the problematic area with the handling of big data. Enforcement of the data centric procedures The data visibility associated with the operating system Encapsulation of the cryptographic techniques Access control The authorised and authenticated users should be provided with the accessible gateway of data for using it. The use of authentication protocol is the major concern of the big Data management system. The precision of data should be taken into consideration for enforcing the sharing of data. Validation and filtering procedures used for input data at end points Validation and filtering of end point data Monitoring of the security issues associated with the real time management system The multiple sources used for collecting the data requires special attention of security The checking of validity of the data is the major security concern Provenance of data The use of metadata creates the complexity of data transfer in the big data environment. Identification of dependencies is the critical scenario for the management of big data The confidentiality of the big data application is the complex and critical region for handling Impact of the security issues: The impact of the security issues associated with the big data management is listed below: Security associated with the parallel programming distributed frameworks Storing of data on the non-relational databases Logging procedures used for data storage In carrying out the validation and filtering process with the end data management system Monitoring procedures used for real time infrastructure Scalability and privacy issues associated with data analytics and mining techniques Enforcement of the data centric storage system Access control used on managing granular data (Cloud security alliance, 2012) Auditing and logging procedures used on granular data Provenance of the data management Proposed Security framework for managing big data: The security of the big data can be managed with the implementation of the proposed framework in the working curriculum of the organization used for handling the big data (Yosepu, 2015). The security infrastructure of the big data depends on five components which are classified as below: Management of the big data associated with the enterprise Identification and accessing of the big data on demand of the user Procedures used for protecting privacy of data during sharing or data retrieval Security procedures used for network handling Proposed infrastructure for managing integrity of the big data of the organization These major divisions are divided into sub modules for handling the big data securely over the web which can be classified from the diagram below: The management of big data security issues depends on restricting unauthorised accessing of data, accountability, development of the balance between activities and network approach (Gahi, 2013). The focus should be given on accessing data from the central database to overcome the possibility of critical attack. The security should be provided to the sensitive data to achieve the integrity between the associated data. The data redaction is capable of providing external level data security. The time management techniques should be used for fetching the data from the central database. The process of tokenization is used for accessing data services from the network. Fully homo-morphic encryption is the most recommendable solution for the smooth functioning of big data available on the cloud network. The decryption procedures are followed at the receiver end to obtain the plain text. The trustworthy data should be stored on the network. The ethical consideration should be given preference to store the data over the network (Singh, 2016). The access controls should be provided for fetching the data in the authorised manner. The restriction should be provided on accessing the information from the cloud network. The stored data on the network should be possessed with authorisation techniques to periodically performing the auditing of the security procedures. The authentication protocols should be used for accessing the information from the central database. The sensitivity of the information should be kept confidential with the use of encryption and cryptographic procedures. The communication procedures should be used for ensuring the sensitivity and integrity of the big data of the enterprise. The threat intelligence should be used for monitoring the security procedures for real time system. The granularity of the data can be achieved with the use of attribute based encryption procedures. The security concern should be taken under consideration for monitoring the leak ages of the sensitive information (Gaddam, 2015). The risk identification and risks assessment matrix should be prepared to provide strategic and tactical solution to the organisation to overcome the situation of complexity arises with the management of big data. The ethical standards and policies should be used to restrict the hacker from carrying over hacking activities. The data should be stored in the structured manner to enhance the retrieval of the information on the demand of the user. Conclusion: The management of security problems associated with the handling of big data involves the inclusion of encryption policies, detection by using honey pot, and logging technique. The increase in the volume of data available on the network will results into the birth of security and privacy challenges associated with the streamlining of the data. The management of big data security issues depends on restricting unauthorised accessing of data, accountability, development of the balance between activities and network approach. The security of the big data can be managed with the implementation of the proposed framework in the working curriculum of the organization used for handling the big data. References: Cloud security alliance. (2012).Top 10 big data security and privacy challenges. Retrieved fromhttps://www.isaca.org/Groups/Professional-English/big-data/GroupDocuments/Big_Data_Top_Ten_v1.pdf Gaddam, A. (2015).Securing your big data environment. Retrieved from https://www.blackhat.com/docs/us-15/materials/us-15-Gaddam-Securing-Your-Big-Data-Environment-wp.pdf Gahi, Y. (2013).Big data analytics: Security and privacy challenges. Retrieved from https://ieeexplore.ieee.org/document/7543859/ Moreno, J. (2016).Main issues in big data security. Retrieved from https://www.google.co.in/url?sa=trct=jq=privacy%20and%20security%20issues%20associated%20with%20big%20datasource=webcd=7cad=rjauact=8ved=0ahUKEwirztOmyrjWAhUDpo8KHW6cBJIQFghSMAYurl=https://www.mdpi.com/1999-5903/8/3/44/pdfusg=AFQjCNFHsoHVfDYme1RD4ap8g9yIPe5Eqw Moura, J. (2013).Security and privacy issues with Big data. Retrieved from https://arxiv.org/ftp/arxiv/papers/1601/1601.06206.pdf Munaye, Y. (2016).Big data security issues, challenges and future scope. Retrieved from https://www.iaeme.com/MasterAdmin/uploadfolder/IJCET_07_04_002-2/IJCET_07_04_002-2.pdf Neves, P. (2016).Big data in cloud computing: Features and issues. Retrieved from https://acme.able.cs.cmu.edu/pubs/uploads/pdf/IoTBD_2016_10.pdf Parms, J. (2013).Emerging big data scenarios has caused privacy and security concerns. https://www.business.com/articles/privacy-and-security-issues-in-the-age-of-big-data/ Schmitt, C. (2013).Security and privacy in the era of big data. Retrieved from https://www.renci.org/wp-content/uploads/2014/02/0213WhitePaper-SMW.pdf Sinanc, D. (2015).A survey on security and privacy issues in big data. Retrieved from https://www.researchgate.net/publication/300413833_A_survey_on_security_and_privacy_issues_in_big_data Singh, R. (2016).Challenges and security issues in big data analysis. Retrieved from https://www.ijirset.com/upload/2016/january/32_Challenges.pdf Wayne, A. (2013).Securing the big data life cycle. Retrieved from https://files.technologyreview.com/whitepapers/Oracle-Securing-the-Big-Data-Life-Cycle.pdf Yosepu, C. (2015).A study on security and privacy in big data processing. Retrieved from https://www.ijircce.com/upload/2015/december/41_A_Study.pdf