journal6 ›› 2014, Vol. 35 ›› Issue (6): 38-41.DOI: 10.3969/j.issn.1007-2985.2014.06.010

• Computer • Previous Articles     Next Articles

Methods of Duplication Screening for ASP Mass Data Storage:A Case Study of Enrollment Information Storage in Xiangxi Vocational and Technical College for Nationalities

 GONG  Shu   

  1.  (XiangXi Vocational and Technical College For Nationalities,Jishou 416000,Hunan China)
  • Online:2014-11-25 Published:2014-11-27

Abstract: When a database is established,judgment on data duplication is crucial for its administration,which will be difficult without accurate keywords for reference.The commonly used methods-Hash technology,fixed-sized partition detection technology,sliding block technology,content-defined chunking detection technology,and fingerprint data exploitation,require a large amount of processing time for the detection and removal of duplication.This paper describes the ASP mass data storage method and duplication screening method,and verifies the robustness and validity of these methods.It is shown that the heavy workload of database management for operators can be greatly reduced.

Key words: duplication removal, data cleaning, data check, screening and storage, data warehouse, data export

WeChat e-book chaoxing Mobile QQ