2024年1月23日发(作者：)

The development and tendency of Big Data

Abstract: "Big Data" is the most popular IT word after the "Internet of things" and

"Cloud computing". From the source, development, status quo and tendency of big

data, we can understand every aspect of it. Big data is one of the most important

technologies around the world and every country has their own way to develop the

technology.

Key words: big data; IT; technology

1 The source of big data

Despite the famous futurist Toffler propose the conception of “Big Data” in 1980,

for a long time, because the primary stage is still in the development of IT industry

and uses of information sources, “Big Data” is not get enough attention by the people

in that age[1].

2 The development of big data

Until the financial crisis in 2008 force the IBM ( multi-national corporation of IT

industry) proposing conception of “Smart City” and vigorously promote Internet of

Things and Cloud computing so that information data has been in a massive growth

meanwhile the need for the technology is very urgent. Under this condition, some

American data processing companies have focused on developing large-scale

concurrent processing system, then the “Big Data” technology become available

sooner and Hadoop mass data concurrent processing system has received wide

attention. Since 2010, IT giants have proposed their products in big data area. Big

companies such as EMC、HP、IBM、Microsoft all purchase other manufacturer

relating to big data in order to achieve technical integration[1]. Based on this, we can

learn how important the big data strategy is. Development of big data thanks to some

big IT companies such as Google、Amazon、China mobile、Alibaba and so on, because

they need a optimization way to store and analysis data. Besides, there are also

demands of health systems、geographic space remote sensing and digital media[2].

3 The status quo of big data

Nowadays America is in the lead of big data technology and market application.

USA federal government announced a “Big Data’s research and development” plan in

March,2012, which involved six federal government department the National Science

Foundation, Health Research Institute, Department of Energy, Department of Defense,

Advanced Research Projects Agency and Geological Survey in order to improve the

ability to extract information and viewpoint of big data[1]. Thus, it can speed science

and engineering discovery up, and it is a major move to push some research

institutions making innovations.

The federal government put big data development into a strategy place, which has

a big impact on every country. At present, many big European institutions is still at

the primary stage to use big data and seriously lack technology about big data. Most

improvements and technology of big data are come from America. Therefore, there

are kind of challenges of Europe to keep in step with the development of big data. But,

in the financial service industry especially investment banking in London is one of the

earliest industries in Europe. The experiment and technology of big data is as good as

the giant institution of America. And, the investment of big data has been maintained

promising efforts. January 2013, British government announced 1.89 million pound

will be invested in big data and calculation of energy saving technology in earth

observation and health care[3].

Japanese government timely takes the challenge of big data strategy. July 2013,

Japan’s communications ministry proposed a synthesize strategy called “Energy ICT

of Japan” which focused on big data application. June 2013, the abe cabinet formally

announced the new IT strategy----“The announcement of creating the most advanced

IT country”. This announcement comprehensively expounded that Japanese new IT

national strategy is with the core of developing opening public data and big data in

2013 to 2020[4].

Big data has also drawn attention of China government.《Guiding opinions of the

State Council on promoting the healthy and orderly development of the Internet of

things》 promote to quicken the core technology including sensor network、intelligent

terminal、big data processing、intelligent analysis and service integration. December

2012, the national development and reform commission add data analysis software

into special guide, in the beginning of 2013 ministry of science and technology

announced that big data research is one of the most important content of “973

program”[1]. This program requests that we need to research the expression, measure

and semantic understanding of multi-source heterogeneous data, research modeling

theory and computational model, promote hardware and software system architecture

by energy optimal distributed storage and processing, analysis the relationship of

complexity、calculability and treatment efficiency[1]. Above all, we can provide theory

evidence for setting up scientific system of big data.

4 The tendency of big data

4.1 See the future by big data

In the beginning of 2008, Alibaba found that the whole number of sellers were on

a slippery slope by mining analyzing user-behavior data meanwhile the procurement

to Europe and America was also glide. They accurately predicting the trend of world

economic trade unfold half year earlier so they avoid the financial crisis[2]. Document

[3] cite an example which turned out can predict a cholera one year earlier by mining

and analysis the data of storm, drought and other natural disaster[3].

4.2 Great changes and business opportunities

With the approval of big data values, giants of every industry all spend more

money in big data industry. Then great changes and business opportunity comes[4].

In hardware industry, big data are facing the challenges of manage, storage and

real-time analysis. Big data will have an important impact of chip and storage industry,

besides, some new industry will be created because of big data[4].

In software and service area, the urgent demand of fast data processing will bring

great boom to data mining and business intelligence industry.

The hidden value of big data can create a lot of new companies, new products,

new technology and new projects[2].

4.3 Development direction of big data

The storage technology of big data is relational database at primary. But due to

the canonical design, friendly query language, efficient ability dealing with online

affair, Big data dominate the market a long term. However, its strict design pattern, it

ensures consistency to give up function, its poor expansibility these problems are

exposed in big data analysis. Then, NoSQL data storage model and Bigtable propsed

by Google start to be in fashion[5].

Big data analysis technology which uses MapReduce technological frame

proposed by Google is used to deal with large scale concurrent batch transaction.

Using file system to store unstructured data is not lost function but also win the

expansilility. Later, there are big data analysis platform like HAVEn proposed by HP

and Fusion Insight proposed by Huawei . Beyond doubt, this situation will be

continued, new technology and measures will come out such as next generation data

warehouse, Hadoop distribute and so on[6].

Conclusion

This paper we analysis the development and tendency of big data. Based on this,

we know that the big data is still at a primary stage, there are too many problems need

to deal with. But the commercial value and market value of big data are the direction

of development to information age.忽略此处..

[1] Li Chunwei, Development report of China’s E-Commerce enterprises, Beijing , 2013,

pp.268-270

[2] Li Fen, Zhu Zhixiang, Liu Shenghui, The development status and the problems of large data,

Journal of Xi’an University of Posts and Telecommunications, 18 volume, pp. 102-103,sep.2013

[3] Kira Radinsky, Eric Horivtz, Mining the Web to Predict Future Events[C]//Proceedings of the

6th ACM International Conference on Web Search and Data Mining, WSDM 2013: New York:

Association for Computing Machinery,2013,pp.255-264

[4] Chapman A, Allen M D, Blaustein B. It’s About the Data: Provenance as a Toll for Assessing

Data Fitness[C]//Proc of the 4th USENIX Workshop on the Theory and Practice of Provenance,

Berkely, CA: USENIX Association, 2012:8

[5] Li Ruiqin, Zheng Janguo, Big data Research: Status quo, Problems and Tendency[J],Network

Application,Shanghai,1994,pp.107-108

[6] Meng Xiaofeng, Wang Huiju, Du Xiaoyong, Big Daya Analysis: Competition and Survival of

RDBMS and ManReduce[J], Journal of software, 2012,23(1): 32-45