科學(xué)計(jì)算科研數(shù)據(jù)分析【免費(fèi)下載鏈接】astropyAstronomy and astrophysics core library項(xiàng)目地址https://gitcode.com/gh_mirrors/as/astropy點(diǎn)擊查看免費(fèi)下載本文基于 astropy 官方文檔 Custom Cosmology I/O formats 展開講解如何向 astropy Cosmology 的統(tǒng)一 I/O 框架注冊自定義格式一是面向內(nèi)存對象轉(zhuǎn)換的to_format/from_formatconvert_registry二是面向文件讀寫的read/writereadwrite_registry。讀完后你可以照著文中完整示例為任意數(shù)據(jù)形態(tài)Row、JSON 等實(shí)現(xiàn)可復(fù)用的 Cosmology 序列化/反序列化并理解其底層注冊、識(shí)別與回環(huán)驗(yàn)證機(jī)制?？蚣芨庞[兩套獨(dú)立的注冊表astropy.cosmology.io包提供了統(tǒng)一接口來讀取、寫出和轉(zhuǎn)換Cosmology對象。用戶入口是Cosmology類上的四個(gè)方法見 astropy/cosmology/io.py 的模塊說明Cosmology.read/Cosmology.write從文件讀入 / 寫出文件Cosmology.from_format/Cosmology.to_format在 Python 對象如 dict、Table與Cosmology之間轉(zhuǎn)換。這四個(gè)方法背后各掛著一套自定義注冊表而不是Table等使用的全局注冊表# astropy/cosmology/_src/io/connect.py readwrite_registry io_registry.UnifiedIORegistry() # L36read/write 用 convert_registry io_registry.UnifiedIORegistry() # L156from/to_format 用在 astropy/cosmology/_src/core.py 中它們以類變量形式綁定到Cosmologyfrom_format: ClassVar UnifiedReadWriteMethod(CosmologyFromFormat) to_format: ClassVar UnifiedReadWriteMethod(CosmologyToFormat) read: ClassVar UnifiedReadWriteMethod(CosmologyRead) write: ClassVar UnifiedReadWriteMethod(CosmologyWrite)因此為某個(gè)格式注冊自定義解析器時(shí)需要拿到對應(yīng)的注冊表Cosmology.from_format.registry即convert_registry——用于to/from_formatCosmology.read.registry即readwrite_registry——用于read/write。每注冊一個(gè)格式通常要提供三個(gè)函數(shù)reader解析為Cosmology、writer從Cosmology輸出、identifier自動(dòng)識(shí)別格式。注冊表本身的 API 位于 astropy/io/registry通用框架說明見 docs/io/registry.rst。一個(gè)值得注意的機(jī)制是CosmologyRead/CosmologyFromFormat在被子類調(diào)用時(shí)例如FlatLambdaCDM.read(...)會(huì)自動(dòng)注入關(guān)鍵字參數(shù)cosmology該子類并校驗(yàn)用戶顯式傳入的cosmology參數(shù)必須與該類或其__qualname__一致否則拋出ValueError見 astropy/cosmology/_src/io/connect.py整個(gè)注冊調(diào)用還會(huì)包在add_enabled_units(cu)上下文中使 astropy 宇宙學(xué)單位如km / (Mpc s)可用。自定義 to/from_format 格式convert_registry官方文檔以astropy.table.Row為例演示了一個(gè)完整的 |Cosmology| ? |Row| 轉(zhuǎn)換器。文檔特意只手寫 mapping 級(jí)別的邏輯而不直接復(fù)用內(nèi)置解析器以便展示最完整的模式。注意真實(shí)的內(nèi)置實(shí)現(xiàn)就在 astropy/cosmology/_src/io/builtin/row.py其from_row的寫法與下面示例幾乎一致同樣是把 Row 轉(zhuǎn)成 dict 后交給from_mapping。1. 編寫 readerRow → Cosmology該函數(shù)應(yīng)接收 1 個(gè)位置參數(shù)行對象和 2 個(gè)關(guān)鍵字參數(shù)move_to_meta多余字段如何處理與cosmology使用哪個(gè)Cosmology類。關(guān)于元數(shù)據(jù)處理的詳細(xì)說明可以通過Cosmology.from_format.help(mapping)查看。 import copy from astropy.cosmology import Cosmology def from_table_row(row, *, move_to_metaFalse, cosmologyNone): ... # get name from column ... name row[name] if name in row.columns else None ... meta copy.deepcopy(row.meta) ... # turn row into mapping (dict of the arguments) ... mapping dict(row) ... mapping[name] name ... mapping.setdefault(cosmology, meta.pop(cosmology, None)) ... mapping[meta] meta ... # build cosmology from map ... return Cosmology.from_format(mapping, move_to_metamove_to_meta, ... cosmologycosmology)關(guān)鍵設(shè)計(jì)在于reader 并不自己解析每個(gè)物理參數(shù)而是把 Row 拼回一個(gè) mapping再委托給內(nèi)置的mapping格式Cosmology.from_format(mapping, ...)。move_to_meta與cosmology兩個(gè)參數(shù)原樣透傳保證行為與內(nèi)置格式一致。對照內(nèi)置實(shí)現(xiàn) astropy/cosmology/_src/io/builtin/mapping.py 中的from_mapping可以看到其核心邏輯是按rename映射重命名鍵并通過_rename_map保證重命名后的鍵不與原鍵沖突沖突會(huì)拋出ValueError從cosmology參數(shù)或 mapping 中的cosmology字段確定目標(biāo)類字符串會(huì)通過_COSMOLOGY_CLASSES解析為類用inspect.signature(cosmology)bind_partialapply_defaults把類簽名中的默認(rèn)值填進(jìn)來再逐個(gè)綁定 mapping 中出現(xiàn)的參數(shù)——這就是用具體子類解析時(shí)缺失參數(shù)會(huì)被類默認(rèn)值補(bǔ)齊的底層機(jī)制剩余多余參數(shù)若構(gòu)造函數(shù)有**kwargs則收入其中否則在move_to_metaTrue時(shí)合并進(jìn)meta顯式 metadata 優(yōu)先否則拋出TypeError: there are unused parameters ...。然后注冊到convert_registry convert_registry.register_reader(astropy.row, Cosmology, from_table_row)2. 編寫 writerCosmology → Rowwriter 只需要Cosmology對象和一個(gè)*args用來吸收astropy.io.registry.UnifiedReadWriteCosmology.to_format的底層實(shí)現(xiàn)見 connect.py傳遞的多余信息 from astropy.table import QTable def to_table_row(cosmology, *args): ... p cosmology.to_format(mapping, cosmology_as_strTrue) ... meta p.pop(meta) ... # package parameters into lists for Table parsing ... params {k: [v] for k, v in p.items()} ... return QTable(params, metameta)[0] # return row convert_registry.register_writer(astropy.row, Cosmology, to_table_row)兩個(gè)細(xì)節(jié)值得注意to_format(mapping, cosmology_as_strTrue)會(huì)把類對象換成類名如FlatLambdaCDM使輸出可以放進(jìn) Table 的str13列。參數(shù)語義可對照 mapping.py 中to_mapping的文檔所有參數(shù)值被包進(jìn)單元素列表{k: [v] for ...}因?yàn)?Table 列必須是序列最后用QTable(...)[0]取出第一行即Row。這與內(nèi)置to_rowbuiltin/row.py內(nèi)部調(diào)to_table后返回table[0]的路徑一致。3. 編寫 identifier 并注冊identifier 用于在用戶不指定format時(shí)自動(dòng)識(shí)別格式。簽名是(origin, format, *args, **kwargs)origin為read或write對convert_registry而言待識(shí)別對象位于args[1]args[0]是目標(biāo)類 from astropy.cosmology import Cosmology from astropy.table import Row def row_identify(origin, format, *args, **kwargs): ... Identify if object uses the Table format. ... if origin read: ... return isinstance(args[1], Row) and (format in (None, astropy.row)) ... return False convert_registry.register_identifier(astropy.row, Cosmology, row_identify)4. 回環(huán)驗(yàn)證注冊完成后即可在Cosmology.from_format/Cosmology.to_format中使用 from astropy.cosmology import Planck18 row Planck18.to_format(astropy.row) row Row index0 cosmology name H0 Om0 Tcmb0 Neff m_nu Ob0 km / (Mpc s) K eV str13 str8 float64 float64 float64 float64 float64[3] float64 ------------- -------- ------------ ------- ------- ------- ----------- ------- FlatLambdaCDM Planck18 67.66 0.30966 2.7255 3.046 0.0 .. 0.06 0.04897 cosmo Cosmology.from_format(row) cosmo Planck18 # test it round-trips Truecosmo Planck18為True說明整個(gè)往返鏈路Cosmology → mapping → QTable/Row → mapping → Cosmology無信息丟失。測試目錄 astropy/cosmology/_src/tests/io/ 中對內(nèi)置 row 格式也有等價(jià)的回環(huán)測試可以參考。自定義 read/write 文件格式readwrite_registry文件級(jí)格式與對象級(jí)格式的區(qū)別在于reader 接收文件路徑或文件對象writer 負(fù)責(zé)落盤。官方文檔給出了一個(gè)完整的 |Cosmology| ? JSON 序列化器示例并且復(fù)用已注冊的 mapping 格式來大幅簡化實(shí)現(xiàn)——這是推薦做法。1. 編寫 readerJSON → Cosmology函數(shù)接收 1 個(gè)位置參數(shù)文件對象或文件路徑其余**kwargs透傳給Cosmology.from_format由它處理元數(shù)據(jù)和目標(biāo)Cosmology類細(xì)節(jié)同Cosmology.from_format.help(mapping) import json, os import astropy.units as u from astropy.cosmology import Cosmology from astropy.cosmology.io import readwrite_registry def read_json(filename, **kwargs): ... # read file, from path-like or file-like ... if isinstance(filename, (str, bytes, os.PathLike)): ... with open(filename, r) as file: ... data file.read() ... else: # file-like : this also handles errors in dumping ... data filename.read() ... mapping json.loads(data) # parse json mappable to dict ... # deserialize Quantity ... for k, v in mapping.items(): ... if isinstance(v, dict) and value in v and unit in v: ... mapping[k] u.Quantity(v[value], v[unit]) ... for k, v in mapping.get(meta, {}).items(): # also the metadata ... if isinstance(v, dict) and value in v and unit in v: ... mapping[meta][k] u.Quantity(v[value], v[unit]) ... return Cosmology.from_format(mapping, **kwargs) readwrite_registry.register_reader(json, Cosmology, read_json)這段代碼的三個(gè)要點(diǎn)兼容路徑與文件對象兩種輸入文件對象分支的注釋指出它順便處理寫出時(shí) dump 的錯(cuò)誤u.Quantity不兼容原生 JSON反序列化約定為{value: ..., unit: ...}結(jié)構(gòu)主參數(shù)和meta中的 Quantity 都要還原解析出 dict 后不再自行構(gòu)造Cosmology而是交給Cosmology.from_format(mapping, **kwargs)——**kwargs中的cosmology/move_to_meta/rename等參數(shù)由 mapping 格式統(tǒng)一處理。2. 編寫 writerCosmology → JSONwriter 需要Cosmology對象與文件對象/路徑并需要一個(gè)布爾標(biāo)志overwrite來定義覆蓋已有文件的行為 def write_json(cosmology, file, *, overwriteFalse, **kwargs): ... data cosmology.to_format(mapping, cosmology_as_strTrue) # start by turning into dict ... # serialize Quantity ... for k, v in data.items(): ... if isinstance(v, u.Quantity): ... data[k] {value: v.value.tolist(), unit: str(v.unit)} ... for k, v in data.get(meta, {}).items(): # also serialize the metadata ... if isinstance(v, u.Quantity): ... data[meta][k] {value: v.value.tolist(), unit: str(v.unit)} ... ... if isinstance(file, (str, bytes, os.PathLike)): ... # check that file exists and whether to overwrite. ... if os.path.exists(file) and not overwrite: ... raise IOError(f{file} exists. Set overwrite to write over.) ... with open(file, w) as write_file: ... json.dump(data, write_file) ... else: ... json.dump(data, file) readwrite_registry.register_writer(json, Cosmology, write_json)注意v.value.tolist()把 ndarray 轉(zhuǎn)成純 Python 列表避免json.dump對 numpy 類型報(bào)錯(cuò)overwriteFalse默認(rèn)時(shí)若文件已存在則拋錯(cuò)這是文件級(jí) writer 的常見契約。倉庫中有一份與文檔示例幾乎同構(gòu)、且在 CI 中持續(xù)運(yùn)行的實(shí)現(xiàn)——單元測試 astropy/cosmology/_src/tests/io/test_json.py它的read_json/write_json/json_identify三個(gè)函數(shù)與文檔示例一一對應(yīng)測試版用Path與OSError并把 Quantity 解析包進(jìn)u.add_enabled_units(cu.redshift)上下文以支持紅移單位還額外驗(yàn)證了用寫入時(shí)同一子類讀取部分信息文件時(shí)缺失參數(shù)會(huì)由類默認(rèn)值補(bǔ)齊test_readwrite_json_subclass_partial_info。這可以作為自己實(shí)現(xiàn)文件級(jí)格式時(shí)的參考模板。3. identifier 與回環(huán)驗(yàn)證 def json_identify(origin, filepath, fileobj, *args, **kwargs): ... Identify if object uses the JSON format. ... return filepath is not None and filepath.endswith(.json) readwrite_registry.register_identifier(json, Cosmology, json_identify)文件級(jí) identifier 的簽名是(origin, filepath, fileobj, *args, **kwargs)按 astropy/io/registry/base.py 中register_identifier的說明filepath與fileobj任一可能為None兩者都為None時(shí) identifier 只能依賴args。按文件擴(kuò)展名判斷是最簡單的可靠策略。注冊后即可完成文件級(jí)讀寫回環(huán) import tempfile from astropy.cosmology import Planck18 file tempfile.NamedTemporaryFile() Planck18.write(file.name, formatjson, overwriteTrue) with open(file.name) as f: f.readlines() [{cosmology: FlatLambdaCDM, name: Planck18, H0: {value: 67.66, unit: km / (Mpc s)}, Om0: 0.30966, ... cosmo Cosmology.read(file.name, formatjson) file.close() cosmo Planck18 # test it round-trips True輸出的 JSON 中可以看到cosmology以字符串類名出現(xiàn)、H0被展開為{value: 67.66, unit: km / (Mpc s)}與 writer 的序列化約定完全對應(yīng)。注冊表 API 細(xì)節(jié)與清理注冊與解除注冊從 astropy/io/registry/core.py 可以看到register_reader/register_writer的完整簽名registry.register_reader(data_format, data_class, function, forceFalse, priority0) registry.register_writer(data_format, data_class, function, forceFalse, priority0) registry.register_identifier(data_format, data_class, identifier, forceFalse) # base.py L213重復(fù)注冊同一(format, class)組合且forceFalse時(shí)會(huì)拋出IORegistryErroralready definedpriority用于在多個(gè)候選格式都匹配時(shí)擇優(yōu)數(shù)值越高越優(yōu)先unregister_reader/unregister_writer/unregister_identifier用于移除注冊若格式不存在同樣拋IORegistryErrorget_reader(format, class)可查詢當(dāng)前注冊的函數(shù)未命中時(shí)錯(cuò)誤信息中會(huì)附帶該類的可用格式表_get_format_table_str。官方文檔示例在結(jié)尾特意清理了注冊狀態(tài)避免示例污染 doctest 環(huán)境 from astropy.io.registry import IORegistryError readwrite_registry.unregister_reader(json, Cosmology) readwrite_registry.unregister_writer(json, Cosmology) readwrite_registry.unregister_identifier(json, Cosmology) try: ... readwrite_registry.get_reader(json, Cosmology) ... except IORegistryError: ... pass這一注冊—測試—注銷的三段式與 test_json.py 中 fixture 的 setup/teardown 寫法注冊時(shí)加forceTrueyield后逐一unregister_*一致是寫測試時(shí)的最佳實(shí)踐。查看可用格式注冊表會(huì)把注冊結(jié)果反映到方法的help()與list_formats()中。當(dāng)前內(nèi)置格式摘自 docs/cosmology/io/index.rst Cosmology.read.list_formats() Format Read Write Auto-identify ---------- ---- ----- ------------- ascii.ecsv Yes Yes Yes ascii.html Yes Yes Yes ascii.mrt Yes Yes Yes Cosmology.write.list_formats() Format Read Write Auto-identify ----------- ---- ----- ------------- ascii.ecsv Yes Yes Yes ascii.html Yes Yes Yes ascii.latex No Yes Yes ascii.mrt Yes Yes Yes Cosmology.to_format.list_formats() Format Read Write Auto-identify ----------------- ---- ----- ------------- astropy.cosmology Yes Yes Yes astropy.model Yes Yes Yes astropy.row Yes Yes Yes astropy.table Yes Yes Yes mapping Yes Yes Yes yaml Yes Yes NoAuto-identify 列即為是否注冊了 identifier如yaml只有 Yes/Yes/No說明它必須顯式傳formatyaml。你注冊的第三方格式會(huì)同樣出現(xiàn)在這兩張表中。實(shí)踐要點(diǎn)小結(jié)先想好復(fù)用哪一層能用內(nèi)置mapping格式做膠水的就不要直接手寫參數(shù)解析——兩個(gè)官方示例Row、JSON都是這樣實(shí)現(xiàn)的行為自動(dòng)與move_to_meta、cosmology、rename等語義保持一致兩套注冊表不要混用對象級(jí)格式注冊到convert_registryCosmology.from_format.registry文件級(jí)格式注冊到readwrite_registryCosmology.read.registry注冊錯(cuò)表會(huì)導(dǎo)致list_formats中出現(xiàn)卻調(diào)不到identifier 決定自動(dòng)識(shí)別能力文件級(jí)建議用擴(kuò)展名等強(qiáng)信號(hào)JSON 例對象級(jí)建議用isinstance判斷Row 例并在format已顯式指定時(shí)放行Quantity 需要自定義編解碼u.Quantity與 JSON 等文本格式不兼容{value: ..., unit: ...}約定 遍歷meta是兩個(gè)官方示例共同采用的方案以回環(huán)斷言收尾Cosmology.from_format(cosmo.to_format(...)) cosmo或文件級(jí)的Cosmology.read是驗(yàn)證實(shí)現(xiàn)正確性的最簡方式官方示例與 astropy/cosmology/_src/tests/io/ 下的測試均以此為標(biāo)準(zhǔn)動(dòng)作。更多背景I/O 總覽見 docs/cosmology/io/index.rst內(nèi)置格式ascii.ecsv、astropy.table、mapping、yaml等的參數(shù)細(xì)節(jié)見 docs/cosmology/io/builtin.rst字段重命名與子類讀取行為見 docs/cosmology/io/details.rst通用 I/O 注冊框架見 docs/io/registry.rst。贊分享科學(xué)計(jì)算科研數(shù)據(jù)分析【免費(fèi)下載鏈接】astropyAstronomy and astrophysics core library項(xiàng)目地址https://gitcode.com/gh_mirrors/as/astropy點(diǎn)擊查看免費(fèi)下載相關(guān)推薦astropy Cosmology 內(nèi)置 I/O 格式詳解Cosmology 對象的轉(zhuǎn)換、讀寫與源碼實(shí)現(xiàn)astropy Cosmology 內(nèi)置 I/O 格式詳解Cosmology 對象的轉(zhuǎn)換、讀寫與源碼實(shí)現(xiàn) 本文基于 astropy 官方文檔 Built in科學(xué)計(jì)算科研數(shù)據(jù)分析astropy 宇宙學(xué)模塊開發(fā)者指南default_cosmology、自定義 Cosmology 與 I/O 互操作astropy 宇宙學(xué)模塊開發(fā)者指南default_cosmology、自定義 Cosmology 與 I/O 互操作 本文基于 astropy 官方開發(fā)者文科學(xué)計(jì)算科研數(shù)據(jù)分析網(wǎng)頁視頻怎么下載貓抓從資源嗅探到 M3U8 合并的 3 步完整流程網(wǎng)頁視頻怎么下載貓抓從資源嗅探到 M3U8 合并的 3 步完整流程 貓抓cat catch是一款瀏覽器資源嗅探擴(kuò)展打開網(wǎng)頁后自動(dòng)列出其中的 MP4、M3音視頻上一篇Dear ImGui 自寫渲染后端如何支持 ImGuiBackendFlags_RendererHasTextures1.92下一篇Gin 開啟 HTTP/2 服務(wù)推送RunTLS 環(huán)境下怎么用 Writer.Pusher 預(yù)推靜態(tài)資源創(chuàng)作聲明：本文部分內(nèi)容由AI輔助生成（AIGC），僅供參考