介紹:就是想實現簡單的ajax上傳數據,但是當數據量較大的時候,問題就一個一個接著來了,其實數據也不是很大,就是csv格式數據 不到5w條數據。大小5M,一開始認為這個很簡單,就是先上傳一下文件,然后讀取一下,存到數據庫就好了,結果,可能我比較菜,弄了半天做出這個功能。環境是linux.在整個調試過程下來,我覺找到問題點才是最重要的,但是一般當一段程序除了問題之后,我們都會覺得,這里有問題,那里有問題,其實這個時候,我們可以這樣,一步一步的來測試,找到問題點在哪,這樣處理問題來,就不會那樣束手無策了--
遇到的問題就從先到后的一一說吧。
問題1 按照我最初的想法,先上傳文件再讀取文件。這里問題就來了,當文件較大的時候上傳較慢,導致客戶看到的操作一直處于等待狀態,不人性化。
處理辦法:我是這樣做的,大神有更好的辦法,求介紹。我先把文件上傳上去,然后把文件存到一個特定的文件夾就叫 import吧 ,然后返回一個這個文件名字。這樣就確保了文件是上傳成功的。并且我可以在他返回名字的這一步用js 給客戶一個提示。然后就是ajax去請求php讀取文件,插入數據庫。可是問題來了。
問題2 當我用ajax去請求php讀取文件并插入數據庫的時候,遇到一個問題,就是ajax請求總是在1min的時候,斷掉。我一想 ,這應該是php的最大執行時間max_execution_time的原因吧,結果我修改為300秒。還是這樣,那我就認為會不會是apache的 最大get時間max_input_time呢,我就在代碼加一個 ini_set 結果,用ini_get 查看max_input_time,用ini_set設置無效,還是60秒,在網上查了很多資料,還是不知道為啥。有大神知道的,請給我回復下。菜鳥先謝過了。那沒辦法,我只能去服務器把php.ini配置修改了。經理說不讓修改的,為了測試,偷偷改了--最后修改回來了。修改之后,測試,還是不行。還是到一分鐘 就執行超時。真的很納悶。不知道什么原因。求指教。那沒辦法。
這種辦法行不通了,對一個5m的文件只能分行讀取了。然后就是對代碼的一通修改,分行讀取是這樣操作的,先ajax請求,然后每次讀取2000條 然后對這2000條數據進行處理,插入數據庫(文章最后介紹一個好用的分行讀取函數)。然后每次ajax執行完,返回一個狀態符,和本次讀取到的行數,然后下次接著讀。知道最后讀取完。這中間還遇到一個問題:就是當我對每一行數據進行查重的時候遇到的,是這樣的,我對得到的內容進行循環,然后查一下每行是否存在,當我判斷$count是否大于0 的時候,當已存在的時候,我用continue,執行下一次循環。但是當我在導入10000條的時候,總是在8000條的時候報錯說 服務器內部錯誤。很悶,不解問什么,結果只能用if else代替了。納悶。一個小提醒:插入數據庫的時候 不要一條一條的插入,最好這樣 inset into aaa(`xx`,`xxx`)values('111','111'),('222','222')。這樣 速度會快很多。
QQWhtU2/Mv7qw3+StN1trZFv+1PVrGc5Xdr6Qss3y15nmjMQEAcPFIEq6TrSUh7Cmx4HhhifvllvFChuZFlJ/KXkPF/uONmmAUcXw0+CIGm3J/USYJ6vdGPOo280WgZxRbX4kjNe6WBFfNbnW7GuUnk9XePCGKX4sa6GSyKk4sHCmBYLLW4/syZ1CPbm0BcJ1P23KcHPdRvUfRQM8P7Zz3RUkSyhNzXudkffhnqZrGnhCiUahtP3/t2iqtDWmyVT6LmYxVnystCVE+Y+3zNvMbJzkAAHSCJOE6VZOE6dLLI3gpGpQ5QDpemLG72oxgNALoL+XB5cZ3RJNad6PqgZwfIfbMJGGXzsv614bdjZomCZWaeDWutdn61iM+vnlFNCmvc1zWjluDxdQaELvPp1059Q+J5T7u0vnGr970vWz3JWtRUS9U7XUuz1//XLaHqq4lwditUe8jy/lv/fzctH5T+kc4MEnY97wZzWgAAFwakoTr9P1JgjPo3/hCa2E4W5JQRNvpvBJEHqElwRH0O504SXCfz5GSBNd91BktTjY19yUcyZ5R6lnVXuf87UcZXJEM+01aEmrOP7vUajPCfOvfaS0Ah7Yk7HneSBIAABeOJOE62bsbKVmBEdPbYvdoYHZVN0ueLj2hdTdS9t/4+UsHJgmV7kb62AM9iDxGd6PqPEj19P23fl6PRI/WLWPB2uA+f0tShtSu82lbzv4nRL+P2nMyXgg9t7ReB2d1uDwf/WTqr3M4Wkf7mhH2a9HXqPa5mm99fbSJ9qQ9vnnNWhLU1oPi1drrQHcjAMClI0m4Psro5C9ClOMEnN2E9G5Fav2lXlTW8yQf3/xFiK/+QJvhVN2/yBDK/+YjZRvmCdbK1GrdrTJK+FXcFeMEZEfzavzn2l4tan9ltrq/fcBxvr0YvJvVTBvDXpPKQGTH+RxWjpXrPlpv+l6O+7IKH/VBwHuv8970Zo+ZL1qMWq45/2K70JuGlHHYKz8uxiW7n6vyeViFk3Wj68DAZQDAxSNJQHeMKVDnu7R2Lh106Gj35fubEbo9/2NgClQAwOUjSUCXjMXUcMOOsqLZLaAZAQBwDUgSAAAAAGhIEvCBWeapbNrjHwAA4IaRJAAAAADQkCQAAAAA0JAkAAAAANCQJAAAAADQkCRcrWnsCeENv3V/JgAAALgtJAnXKhn2yRAAAABwCiQJ14okAQAAACdCknCtSBIAAABwIiQJVykaCNGLOz8NAAAA3CSShGtFSwIAAABOhCThWpEkAAAA4ERIEq4VSQIAAABOhCTharFOAgAAAE6DJAEAAACAhiQBAAAAgIYkAQAAAICGJAEAAACAhiQBAAAAgIYkAQAAAICGJAF2yWTlTf7q/DTwQfC8AQBwUUgSYBHFr+Iu4ye7dL5Lk3WxRdy9itHb3kKSyUrufJTgL4prytn6zU7pWiS/98X9rM1bZn4svN+/tT1Q2OuH0+6Pa3neDrhoR33ekmFfCCGEfn2GX4X4IsQXMdhoF0EI0W7RktlSiPderG0cB+9CvA9mz8P+uxAbrTTb/tPYM7bUbwcAoA2SBFQkaxFv5fdRXAZtal1vbciuOVcN8da/oSQhuhctg/WDjAMx0I7SzXEdz9thjvW8WVY0Hy9Eb5md5+CLP963f61FzxL0b2RuMA7ehViO9+0/30UDYZxG/XYAAJojSYApmaysgZoWfj2+efE2zatv5f6yPtgI0SxB2+Obd5QWCaUcP7nQJCGvkNa+aqPJmR8Locqi9nz7/Sy6ly/1w6ddKqv/tT136VPsxULEgX8vxOc4kf/9HCf6scKeGkp2dVz781a0LXiTv8pWLHmLlUataj5gPG91z2ftc1gN+qPB17JVYbr0tMaEg5IEPUOTScJyLJsU+ovpvv3nBzQmyEYPrXkEAAArkgRUbf27V3G3Ch+17Wr4FY60FoYyyEvWe5OEcFSWnExWDbsJVcrZ+uUZ3lp3I1eNfh6mB1Gla5D5lodAxMJ/2EUyXp/P/Fj4D2ppM18E0UUcd//zprY2pMm2eLv6HDqeE+fzWf8c7k8S8lYF1/71XgZGh6JdOp8tZW4wjTcieNm/v7yY1oi/ZjtJAgCgEZIEOOT1rGoyYK3BbZckqNW3mXXU4HzMcowDPb55F5kktG9J2KV7gnUzsre/5SGQe+bbK8H6NPbaJQknPK76YChB/9bPnw01oDceoQOThH3P4amThGMxmmX2bgcAoCGSBJiiWAnIlFi8DL+StRpRtWxJOLBf0JUmCQfegppg3TFm4ORJwsmO63re0uLRUpsRtBYkS1elxi0Je55DW5KgjEP47u5Gx0KSAAA4EZIEmNSpZoxkQE0Yirit3P745jUYk9B80HNtOWU1c3bON5QkKF16voWfy+mDjhms27obdXJc1/OW3eXRm9mMUNzoxzevWUuC9fmsfw4tQf906Q//zN7bcODyOBBCiOpYgqNp2d1Ing9zHwEAGiBJgJ1ZR2sMHp3vwlH+fdlzYxVO1nlnJNnR3B7/haPm3Y3c5aizssbr2xqWoAwjLgb+PgTFqGJ91lF1zHE/fCoGEAvxOQ7v5f6xHwsR99Wj2Cqbuzpu3XB5pRkh35I/b35cjEt2Pyf253PPc2gN+sPeF9sUqM79o8GJBwC0HLgcDYSw9vUCAKCCJAF2rqANt6MyBWqHx720561t9yFHUtFq8YTWWk6B6mp2AADAgiQB+LiMRc1u/rjNWRdTc3aspuYaon0sLKYGADglkgQAAAAAGpIEXADLfJQtZkcFAADAcZEkAAAAANCQJAAAAADQkCQAAAAA0JAkwK66KBVwOjxvAABcFJIEWKiL4Gaz16srlzVbtqxY9Irgry1l5eOGZr650lkjxlSkXR3X8rwdcNGO+rxZp0CNBl+E+OLl6y6XF8E6BWrtdVsK8W5MVDoO3oV4H8yeh/13ITZaabb9mQIVAHBKJAmoSNbFGrdRXAZtal1vFDcNxaghPkB0L1oG6wepLGrWzXEdz9thjvW8VRdHS4ZfxWCTjheVJMG+f71FzxL0b2RuMA7ehViO9+3fejE1AABaIEmAybX2rRZ+Pb558TbNq2/l/rI+2AjRLEGbOuHpvhaJrPx4bZkUVWncyA4hSx6tfLlntkP+ljbHPdrFzCqkta/aaHLmx0Kosqg9334/i+7lS/3waZfK6n9tz136FHuxEHHg3wvxOU7kfz/HiX6ssKeGkl0d1/68FW0L3uSv8kbLu1a97+7nre75rH0enEH/EZMEY9npabyRucE4eBf9xXTf/vMDGhNko8elr2QHALgEJAmo2vp3r+JuFT5q29XwKxxpLQxlkJes9yYJ4agsOZmsmuUJWaCvlZZso7LM/Hwe3zwZFCZr+a7i9Noet0OuGv08TA+iStcg8y0PgYiF/7CLZLw+n/mx8B/U0mZ+ZT3gjo67/3lTWxvs9939vLmez/rn4dRJwsvA6FC0S+ezpcwNpvFGBC/795cX0xrx12wnSQAANEKSAIe8nlVNBqw1uO2SBMu6aXtWTHOWrxdVJglFfXPe1uEnhxz3KNq3JOzSPcG6Gdnb3/IQyD3z7ZVgfRp77ZKEEx7X+ryl862f3yM1oLffd9fz5np+9j0Pp04SjsVoltm7HQCAhkgSYIpiJSBTgvIy/Mor6YvtbVoStn7LKnxH+Vv/TqsJ3pMktD9ul7egJlh3jBk4eZJwsuO6nrfy3qnNCK77rrylWUvCnueBJAEA8MGRJMCkTjVjJANqwlDEbeX2xzevwZiE5oOeixLsNcFFkFd0MZrXJAmtj9shpUvPt/BzOX3QMYN1W3ejTo7ret7S+U6G8mYzgvW+u5831/NZ/zwcJ0kYB0IIUR1LcDQtuxvJ82HuIwBAAyQJsDPraI3Bo/NdOMq/L3turMLJOu+MJDua2+O/cNS020/RxykfZlB2dlK6P638eCXuXv1JfibxVu7sTf5Sh642P27XlGHExcDfh6AYVazPOqqOOe6HT8UAYiE+x+G93D/2YyHivnoUW2VzV8etGy6vNCNoj0R532VLket5sz+fe56HatAv5z8t9ZZp7f7pfBcNTjwAoOXA5WgghLWvFwAAFSQJsHMFbbgdlSlQOzzupT1vbbsPWfcPe60WT2it5RSormYHAAAsSBKAj8tY1Ozmj9ucdTE1B8diaq4h2sfCYmoAgFMiSQAAAACgIUnABbDMR3n5wwYAAABuFkkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAgAAAAANSQIAAAAADUkCAAAAAA1JAtCBP1p+dX7CAADgQyFJADpAkgAAAC4ZSQLQAZIEAABwyUgSYJdMVt7kr85P40o9D/ub4beaHf7447d///DDv3/rOEnYe54AAOBjIkmARRS/iruMn3R/PpdnthTivRe7Xw1e9I3Pw/67EO+iv5ju0vnujz/++OenH/7++dd/2JOCp3/9rL2aznfpOHgX4n0wk0Upwf1sKYS+pfmnMM6z/nMBAICPgiQBFclaxFv5fRR/V5IQjtZRm+1XZNFzBtPW6nlj4x9//PHH06//625M+OenH/7+VL6WznfpNN7ITGAcvAuxHO8/aDrfpePAlTxY31LzuQAAwMdBkgBTMllVE4Nw9CruVuFknbcwrMLH/NWk2Pha9lB6fPPuyuaIskXCtd14afSmH/rVm/yVTFbi7lXcraP5Tn7vx8WhtaxDvkXcvYrRylOKsnzYYV9UvrwGVfKLnngfzGwvWZoRUmuSUNuY8I9ff/7fX5+qScJyLJsUshYJV/klZ5JgP0/H55r5QgjRD6fVcgAAwA0iSUDV1jfSgPkuzSLvPBZ/fPOK75NtpOyjJhitWhLCUXnEZLJS84QsD5FbkrU8RDJZFeejjqBQv49iLd84opeBvXuPK1i3Jgl//Pbpb1djwm+fzCRhPlvK3GAab2zdmVolCa79HZ+LJAEAgI+FJAEOeb1+EfQbCUDZ4KA3DhyYJFRaGLTGgWRdHUWttXhoO8gkx9LCUHVwS4KDvXo+dSUJcuzBp/3jl/ce2gz6p/FGiHed0kPJeZ4AAAApSQKqoljrSlQE32pNf1rG6Fv/TmsBOLQlYevXVPm3SxK0N56oJcGqZrIgR5IgGxN+/teTOz84LEko2FoSmNQIAADUI0mASZ3aSK2JD0fGUIF1NN+lj29lp//HN89sScjzh2StF2XZHsWvzklX2yQJWjKjDMI+vbrqeWeS0KwxYe/R2yQJ7ZsRxoEQQjCgGQCAD4MkAXbV4cvhaOXHq+oY5Xw88au4y3ZQY3d7tx/H9nLAcfmS2neo7M5UHNRPytLkKdkKOYf66nl3ktCoMeHwo1eShAOaEaKBECK49gmpAABAcyQJsLMmCcZQZijyUcWOHeqShAaNCXtPoHHov+c8rW/xGbIMAMAHQ5KARpRJRc/Xxf+WVBdTqzQmfPrngUnCwYupAQAA2JEkAB2o61p0UEsCAADAEZEkAB0gSQAAAJeMJAEAAACAhiQBAAAAgIYkAQAAAICGJAF2yWTlXNoMuBU851xnAIAVSQIs1EWXs9USyuXPmk6EWqx3dpTgoG49Zrng2gXMzZr83hf3szZvmfmx8H7/1vZAYe92Fi5Ifu+LWIhYtLx0hzCum+U5/4DGwbsQ74OZnKW3nEjXtb296/59Mo0961rjru0AcCtIElCRrEW8ld9HcRk8qXWBtX9iNeeqQdz6F5AkRPfniHTTcSAGpz/K7V0647o5nvMDhCP7wt6u7ZdlGm9kDjAO3oVYjvdtb+v6f59EA+HbPr5rOwDcBpIEmKprLRfbyz/Pj29evE3z6j25v6wvNP6EW/6oP755R6lBVMrxkyMnCcmwLypfXl1l6syX1eGFLOTNt9/Ponv5Uj982qXWGvSn2IuFiAP/XojPcSL/+zlO9GOFvTI08WIhYuE/fAs/y8KDyDifynH1Uw3KKDY7uhBxP/w90N/i/rxNys9Ltp1nppok5MUKYV6EopBmn8t23VzPeTh6FXercFLUcyurjCuV39oPglojXtSUu7
新聞熱點
疑難解答