江西建設(shè)廳官方網(wǎng)站,自動(dòng)發(fā)貨網(wǎng)站建設(shè),營(yíng)銷策劃方案設(shè)計(jì)的技巧,百度廣告競(jìng)價(jià)中石油旗下子公司大文件傳輸系統(tǒng)技術(shù)方案 一、項(xiàng)目背景與需求分析 作為中石油集團(tuán)旗下專注于能源信息化領(lǐng)域的子公司#xff0c;我司長(zhǎng)期服務(wù)于政府及軍工單位#xff0c;在能源管理、安全生產(chǎn)等關(guān)鍵領(lǐng)域積累了豐富的行業(yè)經(jīng)驗(yàn)。本次政府招投標(biāo)項(xiàng)目提出的大文件傳輸需求具有…中石油旗下子公司大文件傳輸系統(tǒng)技術(shù)方案一、項(xiàng)目背景與需求分析作為中石油集團(tuán)旗下專注于能源信息化領(lǐng)域的子公司我司長(zhǎng)期服務(wù)于政府及軍工單位在能源管理、安全生產(chǎn)等關(guān)鍵領(lǐng)域積累了豐富的行業(yè)經(jīng)驗(yàn)。本次政府招投標(biāo)項(xiàng)目提出的大文件傳輸需求具有以下特殊性功能需求支持20GB以上超大文件傳輸支持文件夾結(jié)構(gòu)上傳/下載保持目錄層級(jí)支持?jǐn)帱c(diǎn)續(xù)傳、分片校驗(yàn)、傳輸加密支持傳輸隊(duì)列管理優(yōu)先級(jí)控制兼容性要求主流瀏覽器Chrome/Firefox/Edge/國(guó)產(chǎn)瀏覽器信創(chuàng)國(guó)產(chǎn)化環(huán)境麒麟/統(tǒng)信UOS 飛騰/鯤鵬/龍芯數(shù)據(jù)庫(kù)兼容性SQL Server/達(dá)夢(mèng)/人大金倉(cāng)技術(shù)架構(gòu)約束后端ASP.NET Framework 4.8暫不考慮.NET Core遷移前端ASP.NET WebForms需兼容IE11及現(xiàn)代瀏覽器需提供完整源代碼及知識(shí)產(chǎn)權(quán)特殊需求軍工級(jí)安全要求等保三級(jí)傳輸過(guò)程不可逆加密詳細(xì)的操作審計(jì)日志二、現(xiàn)有方案評(píng)估2.1 已評(píng)估開(kāi)源方案組件名稱優(yōu)點(diǎn)缺點(diǎn)WebUploader成熟度高已停更最后更新2018年不支持信創(chuàng)環(huán)境Fine Uploader商業(yè)級(jí)功能許可證成本高定制開(kāi)發(fā)受限D(zhuǎn)ropzone.js簡(jiǎn)單易用僅支持基礎(chǔ)上傳缺乏大文件處理能力Plupload多后端支持文檔陳舊.NET實(shí)現(xiàn)效率低2.2 核心問(wèn)題信創(chuàng)環(huán)境兼容性現(xiàn)有組件均未針對(duì)國(guó)產(chǎn)CPU架構(gòu)和操作系統(tǒng)進(jìn)行優(yōu)化技術(shù)架構(gòu)限制ASP.NET WebForms的異步處理能力有限安全合規(guī)性開(kāi)源組件缺乏軍工級(jí)加密和審計(jì)功能文件夾支持大多數(shù)組件僅支持單文件上傳三、自研組件技術(shù)方案3.1 架構(gòu)設(shè)計(jì)┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐ │ ASP.NET WebForms │ │ IIS (WebDAV) │ │ WCF/ASP.NET服務(wù) │ │ (前端上傳控件) │←──→│ (靜態(tài)資源代理) │←──→│ (傳輸處理服務(wù)) │ └─────────────────────┘ └─────────────────────┘ └─────────────────────┘ ↑ ↑ ↑ HTML5 File API SignalR實(shí)時(shí)通知 ADO.NET多數(shù)據(jù)庫(kù)適配 (Folder API polyfill) (傳輸進(jìn)度推送) (SQL Server/達(dá)夢(mèng)/人大金倉(cāng))3.2 核心代碼實(shí)現(xiàn)前端實(shí)現(xiàn)ASP.NET WebForms控件// FileTransferControl.ascx.cspublicpartialclassFileTransferControl:System.Web.UI.UserControl{protectedvoidPage_Load(objectsender,EventArgse){if(!IsPostBack){RegisterClientScripts();}}privatevoidRegisterClientScripts(){stringscript // 文件夾選擇兼容性處理IE11現(xiàn)代瀏覽器 function handleFolderSelect(event) { const input event.target; if (input.files input.files.length 0) { processFiles(input.files); } } // 分片上傳核心邏輯 async function uploadFile(file, options) { const chunkSize 10 * 1024 * 1024; // 10MB分片 const chunks Math.ceil(file.size / chunkSize); let uploadedChunks 0; // 計(jì)算文件SM3哈希國(guó)產(chǎn)密碼算法 const fileHash await calculateSM3Hash(file); // 檢查文件是否已存在 const checkResult await $.ajax({ url: FileTransferHandler.ashx?actioncheck, type: POST, data: { fileName: file.name, fileSize: file.size, fileHash } }); if (checkResult.exists) { showProgress(file.name, 100); return; } // 分片上傳 for (let i 0; i chunks; i) { const start i * chunkSize; const end Math.min(start chunkSize, file.size); const chunk file.slice(start, end); const formData new FormData(); formData.append(file, chunk); formData.append(fileId, options.fileId); formData.append(chunkIndex, i); formData.append(totalChunks, chunks); formData.append(fileHash, fileHash); formData.append(relativePath, options.relativePath || ); await $.ajax({ url: FileTransferHandler.ashx?actionupload, type: POST, data: formData, processData: false, contentType: false, xhr: function() { const xhr new XMLHttpRequest(); xhr.upload.addEventListener(progress, function(e) { if (e.lengthComputable) { const progress Math.round( (uploadedChunks * chunkSize e.loaded) / file.size * 100 ); showProgress(file.name, progress); } }, false); return xhr; } }); uploadedChunks; } // 通知服務(wù)器合并 await $.ajax({ url: FileTransferHandler.ashx?actionmerge, type: POST, data: { fileId: options.fileId, fileHash, totalChunks: chunks, originalName: file.name, relativePath: options.relativePath } }); showProgress(file.name, 100); } // SM3哈希計(jì)算通過(guò)Web Worker function calculateSM3Hash(file) { return new Promise((resolve) { const worker new Worker(/Scripts/sm3.worker.js); worker.postMessage({ file }); worker.onmessage (e) resolve(e.data.hash); }); };ScriptManager.RegisterStartupScript(this,GetType(),FileTransferScript,script,true);}protectedvoidbtnUpload_Click(objectsender,EventArgse){// 服務(wù)器端初始化上傳會(huì)話stringsessionIdGuid.NewGuid().ToString();Session[FileTransfer_sessionId]newFileTransferSession{StartTimeDateTime.Now,UserIdUser.Identity.Name,ClientIPRequest.UserHostAddress};hdnSessionId.ValuesessionId;}}后端實(shí)現(xiàn)ASP.NET通用處理程序// FileTransferHandler.ashx.cspublicclassFileTransferHandler:IHttpHandler{privatereadonlyIFileStorageService_storageService;privatereadonlyIAuditLogService_auditLog;publicFileTransferHandler(){// 通過(guò)依賴注入或服務(wù)定位器獲取服務(wù)_storageServiceServiceLocator.Current.GetInstance();_auditLogServiceLocator.Current.GetInstance();}publicvoidProcessRequest(HttpContextcontext){context.Response.ContentTypeapplication/json;try{varactioncontext.Request[action];varsessionIdcontext.Request[sessionId]??context.Request.Cookies[FileTransferSession];varsessioncontext.Session[FileTransfer_sessionId]asFileTransferSession;if(sessionnull){thrownewInvalidOperationException(無(wú)效的傳輸會(huì)話);}switch(action?.ToLower()){casecheck:HandleCheckRequest(context,session);break;caseupload:HandleUploadRequest(context,session);break;casemerge:HandleMergeRequest(context,session);break;casedownload:HandleDownloadRequest(context,session);break;default:thrownewNotSupportedException(不支持的操作);}}catch(Exceptionex){context.Response.StatusCode500;context.Response.Write(JsonConvert.SerializeObject(new{successfalse,messageex.Message}));// 記錄錯(cuò)誤日志_auditLog.LogError(FileTransfer,ex.ToString(),context.Request.UserHostAddress,User.Identity.Name);}}privatevoidHandleCheckRequest(HttpContextcontext,FileTransferSessionsession){varfileNamecontext.Request[fileName];varfileSizelong.Parse(context.Request[fileSize]);varfileHashcontext.Request[fileHash];// 多數(shù)據(jù)庫(kù)適配查詢varexists_storageService.CheckFileExists(fileHash,fileSize);context.Response.Write(JsonConvert.SerializeObject(new{successtrue,exists,fileIdGuid.NewGuid().ToString()}));// 記錄審計(jì)日志_auditLog.LogAccess(FileCheck,${fileName}({fileSize}bytes),context.Request.UserHostAddress,User.Identity.Name);}privatevoidHandleUploadRequest(HttpContextcontext,FileTransferSessionsession){if(context.Request.Files.Count0){thrownewArgumentException(未接收到文件數(shù)據(jù));}varfilecontext.Request.Files[0];varfileIdcontext.Request[fileId];varchunkIndexint.Parse(context.Request[chunkIndex]);vartotalChunksint.Parse(context.Request[totalChunks]);varfileHashcontext.Request[fileHash];varrelativePathcontext.Request[relativePath];// 臨時(shí)存儲(chǔ)路徑按會(huì)話ID分目錄vartempPathPath.Combine(_storageService.TempDirectory,fileId);Directory.CreateDirectory(tempPath);varchunkPathPath.Combine(tempPath,$chunk-{chunkIndex});file.SaveAs(chunkPath);// 記錄分片上傳日志數(shù)據(jù)庫(kù)適配using(vardbDatabaseFactory.CreateDatabase()){db.ExecuteNonQuery(INSERT INTO FileTransferChunks (FileId, ChunkIndex, Size, UploadTime) VALUES (fileId, chunkIndex, size, GETDATE()),fileId,chunkIndex,file.ContentLength);}context.Response.Write(JsonConvert.SerializeObject(new{successtrue,chunkIndex,receivedSizefile.ContentLength}));}// 其他處理方法...}多數(shù)據(jù)庫(kù)適配層// DatabaseFactory.cspublicstaticclassDatabaseFactory{privatestaticreadonlystring_dbTypeConfigurationManager.AppSettings[DatabaseType];publicstaticIDatabaseCreateDatabase(){switch(_dbType?.ToUpper()){caseDM:returnnewDamengDatabaseAdapter();caseKINGBASE:returnnewKingbaseDatabaseAdapter();caseSQLSERVER:default:returnnewSqlServerDatabaseAdapter();}}}publicinterfaceIDatabase:IDisposable{intExecuteNonQuery(stringsql,paramsobject[]parameters);objectExecuteScalar(stringsql,paramsobject[]parameters);DataTableExecuteDataTable(stringsql,paramsobject[]parameters);}// SQL Server實(shí)現(xiàn)publicclassSqlServerDatabaseAdapter:IDatabase{privateSqlConnection_connection;publicSqlServerDatabaseAdapter(){_connectionnewSqlConnection(ConfigurationManager.ConnectionStrings[SqlServer].ConnectionString);_connection.Open();}publicintExecuteNonQuery(stringsql,paramsobject[]parameters){using(varcmdnewSqlCommand(sql,_connection)){cmd.Parameters.AddRange(parameters.Select((p,i)newSqlParameter($param{i},p??DBNull.Value)).ToArray());returncmd.ExecuteNonQuery();}}// 其他方法實(shí)現(xiàn)...publicvoidDispose(){_connection?.Dispose();}}// 達(dá)夢(mèng)數(shù)據(jù)庫(kù)實(shí)現(xiàn)publicclassDamengDatabaseAdapter:IDatabase{privateDmConnection_connection;publicDamengDatabaseAdapter(){_connectionnewDmConnection(ConfigurationManager.ConnectionStrings[Dameng].ConnectionString);_connection.Open();}publicintExecuteNonQuery(stringsql,paramsobject[]parameters){// 達(dá)夢(mèng)特定語(yǔ)法適配sqlsql.Replace(GETDATE(),SYSDATE).Replace([,).Replace(],);using(varcmdnewDmCommand(sql,_connection)){cmd.Parameters.AddRange(parameters.Select((p,i)newDmParameter($:param{i},p??DBNull.Value)).ToArray());returncmd.ExecuteNonQuery();}}// 其他方法實(shí)現(xiàn)...}3.3 信創(chuàng)環(huán)境適配方案國(guó)產(chǎn)密碼算法集成// SM3哈希計(jì)算通過(guò)P/Invoke調(diào)用國(guó)產(chǎn)密碼庫(kù)publicstaticclassSM3Helper{[DllImport(GMSSL.dll,EntryPointSM3_Compute,CharSetCharSet.Ansi)]privatestaticexternintNativeSM3Compute(byte[]input,intinputLen,byte[]output);publicstaticstringComputeHash(Streamstream){constintbufferSize8192;byte[]buffernewbyte[bufferSize];intbytesRead;using(varsm3SM3.Create())// 使用BouncyCastle的SM3實(shí)現(xiàn)或國(guó)產(chǎn)密碼庫(kù){while((bytesReadstream.Read(buffer,0,bufferSize))0){sm3.TransformBlock(buffer,0,bytesRead,null,0);}sm3.TransformFinalBlock(buffer,0,0);returnBitConverter.ToString(sm3.Hash).Replace(-,).ToLower();}}}國(guó)產(chǎn)操作系統(tǒng)文件系統(tǒng)優(yōu)化publicclassKylinFileOptimizer:IFileOptimizer{publicvoidOptimizeFileHandle(FileInfofile){if(IsKylinOS()){// 調(diào)用國(guó)產(chǎn)文件系統(tǒng)特性NativeMethods.SetFileAttribute(file.FullName,FileAttributes.KylinOptimized);}}privateboolIsKylinOS(){returnDirectory.Exists(/etc/kylin)||Directory.Exists(/usr/lib/kylin);}}Web服務(wù)器配置四、實(shí)施路線圖第一階段6周完成核心傳輸功能開(kāi)發(fā)實(shí)現(xiàn)文件夾結(jié)構(gòu)解析與重建完成SQL Server基礎(chǔ)版本第二階段4周達(dá)夢(mèng)/人大金倉(cāng)數(shù)據(jù)庫(kù)適配信創(chuàng)環(huán)境專項(xiàng)優(yōu)化軍工級(jí)安全加固第三階段2周性能調(diào)優(yōu)與壓力測(cè)試操作手冊(cè)與API文檔編寫知識(shí)產(chǎn)權(quán)梳理與源代碼審計(jì)五、預(yù)期收益完全自主可控掌握全部源代碼適應(yīng)軍工項(xiàng)目特殊要求多數(shù)據(jù)庫(kù)支持無(wú)縫切換SQL Server/達(dá)夢(mèng)/人大金倉(cāng)信創(chuàng)環(huán)境優(yōu)化針對(duì)國(guó)產(chǎn)軟硬件環(huán)境深度優(yōu)化安全合規(guī)滿足等保三級(jí)要求集成國(guó)產(chǎn)密碼算法性能提升通過(guò)分片并行傳輸充分利用網(wǎng)絡(luò)帶寬我司已組建由.NET資深專家領(lǐng)銜的專項(xiàng)團(tuán)隊(duì)整合集團(tuán)內(nèi)部密碼技術(shù)資源預(yù)計(jì)在3個(gè)月內(nèi)完成符合軍工標(biāo)準(zhǔn)的大文件傳輸系統(tǒng)開(kāi)發(fā)。該系統(tǒng)不僅滿足當(dāng)前項(xiàng)目需求更可打造為能源行業(yè)信創(chuàng)解決方案的核心組件。將組件復(fù)制到項(xiàng)目中示例中已經(jīng)包含此目錄引入組件配置接口地址接口地址分別對(duì)應(yīng)文件初始化文件數(shù)據(jù)上傳文件進(jìn)度文件上傳完畢文件刪除文件夾初始化文件夾刪除文件列表參考http://www.ncmem.com/doc/view.aspx?ide1f49f3e1d4742e19135e00bd41fa3de處理事件啟動(dòng)測(cè)試啟動(dòng)成功效果數(shù)據(jù)庫(kù)下載示例點(diǎn)擊下載完整示例