2024/11/21-20:02:58.437785 139783438915136 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12121/att-00: fork to child time = 0.008193 sec 2024/11/21-20:02:58.438638 139783438915136 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12121/att-00 2024/11/21-20:02:58.452290 139783438915136 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12121/att-00 2024/11/21-20:02:58.453226 139783438915136 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/21-20:02:58.453489 139783438915136 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/21-20:02:58.508785 139783438915136 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T10.40.08/.rocksdb/job-12121/att-00/MANIFEST-130099 2024/11/21-20:02:58.548342 139783438915136 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T10.40.08/.rocksdb/job-12121/att-00/MANIFEST-130099 succeeded,manifest_file_number is 130099, next_file_number is 145507, last_sequence is 2303559782, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 145348 2024/11/21-20:02:58.548349 139783438915136 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/21-20:02:58.548670 139783438915136 [version_set.cc:5721] Creating manifest 145507 2024/11/21-20:02:58.585897 139783438915136 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12121/att-00: bottommost_level: fake = 1, rpc = 1 2024/11/21-20:02:58.658807 139783438915136 [compaction_job.cc:2437] [default] [JOB 12121] Compacting 1@3 + 136@4 files to L4, score 0.41, subcompactions 1 : 1 2024/11/21-20:02:58.658874 139783438915136 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 12049 Base level 3, inputs: [145505(908KB)], [139342(503KB) 139343(3342KB) 139344(3793KB) 139345(5007KB) 139346(5213KB) 139347(3793KB) 139348(3793KB) 139349(3793KB) 139350(4064KB) 139353(2101KB) 139435(2403KB) 139489(808KB) 139499(586KB) 139575(1694KB) 139578(711KB) 139635(1455KB) 139718(1267KB) 139735(1138KB) 139794(1392KB) 139879(1267KB) 139887(1138KB) 139963(1392KB) 140012(1267KB) 140016(1138KB) 140088(57KB) 140101(1331KB) 140166(1267KB) 140170(1075KB) 140242(825KB) 140233(696KB) 140306(1204KB) 140320(1075KB) 140395(1189KB) 140399(332KB) 140481(2213KB) 140536(1252KB) 140556(395KB) 140636(2150KB) 140705(1645KB) 140763(1505KB) 140756(585KB) 140859(1708KB) 140934(2087KB) 140994(1708KB) 141072(2087KB) 141162(1771KB) 141219(2024KB) 141299(1771KB) 141384(1441KB) 141369(585KB) 141462(1315KB) 141463(458KB) 141528(1961KB) 141614(1378KB) 141610(458KB) 141670(1441KB) 141685(522KB) 141769(1898KB) 141830(1834KB) 141918(1441KB) 141909(522KB) 141989(1378KB) 142009(458KB) 142063(1441KB) 142081(522KB) 142151(1834KB) 142225(1898KB) 142308(1898KB) 142388(1898KB) 142457(1441KB) 142519(915KB) 142607(1378KB) 142598(395KB) 142689(1505KB) 142667(522KB) 142756(1771KB) 142822(2087KB) 142897(1315KB) 142906(395KB) 142992(1505KB) 142979(522KB) 143069(1378KB) 143054(458KB) 143144(1441KB) 143136(522KB) 143197(1378KB) 143215(458KB) 143294(1441KB) 143280(522KB) 143375(1771KB) 143439(1505KB) 143431(522KB) 143507(1378KB) 143515(395KB) 143588(1505KB) 143592(522KB) 143663(1378KB) 143673(395KB) 143766(2087KB) 143838(871KB) 143830(903KB) 143897(1190KB) 143881(710KB) 143974(1190KB) 143961(710KB) 144032(1190KB) 144025(710KB) 144123(1898KB) 144172(1190KB) 144178(710KB) 144257(1190KB) 144288(710KB) 144358(1898KB) 144439(1190KB) 144418(710KB) 144486(1190KB) 144493(710KB) 144575(1190KB) 144569(710KB) 144669(1898KB) 144732(1898KB) 144791(1190KB) 144806(710KB) 144866(1190KB) 144889(710KB) 144931(1190KB) 144952(710KB) 145022(1898KB) 145096(1190KB) 145111(710KB) 145178(1898KB) 145269(1898KB) 145340(1190KB) 145308(710KB) 145384(1190KB) 145393(710KB)] 2024/11/21-20:02:58.665405 139783438915136 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/21-20:02:58.665437 139783438915136 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/21-20:02:58.665464 139783438915136 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/21-20:02:58.665470 139783438915136 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-20:02:58.665475 139783438915136 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/21-20:03:03.831864 139783438915136 [compaction_job.cc:703] job-12121: subcompact[0], size: 0.000000 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12121/att-00 [] 2024/11/21-20:03:03.842835 139783438915136 (Original Log Time 2024/11/21-20:02:58.658947) EVENT_LOG_v1 {"time_micros": 1732190578658894, "job": 12121, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L3": [145505], "files_L4": [139342, 139343, 139344, 139345, 139346, 139347, 139348, 139349, 139350, 139353, 139435, 139489, 139499, 139575, 139578, 139635, 139718, 139735, 139794, 139879, 139887, 139963, 140012, 140016, 140088, 140101, 140166, 140170, 140242, 140233, 140306, 140320, 140395, 140399, 140481, 140536, 140556, 140636, 140705, 140763, 140756, 140859, 140934, 140994, 141072, 141162, 141219, 141299, 141384, 141369, 141462, 141463, 141528, 141614, 141610, 141670, 141685, 141769, 141830, 141918, 141909, 141989, 142009, 142063, 142081, 142151, 142225, 142308, 142388, 142457, 142519, 142607, 142598, 142689, 142667, 142756, 142822, 142897, 142906, 142992, 142979, 143069, 143054, 143144, 143136, 143197, 143215, 143294, 143280, 143375, 143439, 143431, 143507, 143515, 143588, 143592, 143663, 143673, 143766, 143838, 143830, 143897, 143881, 143974, 143961, 144032, 144025, 144123, 144172, 144178, 144257, 144288, 144358, 144439, 144418, 144486, 144493, 144575, 144569, 144669, 144732, 144791, 144806, 144866, 144889, 144931, 144952, 145022, 145096, 145111, 145178, 145269, 145340, 145308, 145384, 145393], "score": 0.414156, "input_data_size": 191525070, "oldest_snapshot_seqno": 2303528954} 2024/11/21-20:03:03.842840 139783438915136 (Original Log Time 2024/11/21-20:03:03.842568) [compaction_job.cc:2022] [default] [JOB 12121] Compacted 1@3 + 136@4 files to L4 => 0 bytes 2024/11/21-20:03:03.842842 139783438915136 (Original Log Time 2024/11/21-20:03:03.842761) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 0 0 0 0] max score 0.00, MB/sec: 37.0 rd, 0.0 wr, level 4, files in(1, 136) out(0 +0 blob) MB in(0.9, 181.8 +0.0 blob) out(0.0 +0.0 blob), read-write-amplify(205.8) write-amplify(0.0) OK, records in: 8983429, records dropped: 8983429 output_compression: Snappy 2024/11/21-20:03:03.842845 139783438915136 (Original Log Time 2024/11/21-20:03:03.842810) EVENT_LOG_v1 {"time_micros": 1732190583842780, "cf": "default", "job": 12121, "event": "compaction_finished", "compaction_time_micros": 5172951, "compaction_time_cpu_micros": 998420, "output_level": 4, "num_output_files": 0, "total_output_size": 0, "num_input_records": 8983429, "num_output_records": 0, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 0, "file_range_sync_nanos": 0, "file_fsync_nanos": 0, "file_prepare_write_nanos": 0, "lsm_state": [0, 0, 0, 0, 0, 0, 0]} 2024/11/21-20:03:03.976238 139783438915136 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732156808,"starts":1732190578,"executesMs":5395,"compactionJobId":12121,"attempt":0,"compactionInputRawBytes":390251199,"compactionInputZipBytes":191525070,"compactionOutputRawBytes":0,"compactionOutputZipBytes":0,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-25cec85300aafe4c6b3cb10aa127c20e-99a2ae7322aff54d-00"}}, response = 2024/11/21-20:03:03.976384 139783438915136 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-21T10.40.08/.rocksdb/job-12121/att-00: olev 4, work 5.395 s, result 9.509 ms, install 0.305 ms, input{raw 372.173 MiB zip 182.653 MiB}