StaticFetcher() Implementation + from_pdb() refactor - #5436
Conversation
|
Docs need to be reupdated, but the code itself feel like it is in a completed state. |
Documentation build overview
10 files changed ·
|
yuxuanzhuang
left a comment
There was a problem hiding this comment.
I think the code is in pretty good shape. I will add more comments related to the code.
For the documentation, it would be helpful to add an overview page at package/doc/sphinx/source/documentation_pages/fetchers_modules.rst that includes:
- Usage examples.
- A list of the available fetchers.
See package/doc/sphinx/source/documentation_pages/converters.rst for an example.
|
|
||
| cache_files = [ | ||
| path | ||
| for path in self.cache_path.rglob("*") |
There was a problem hiding this comment.
Checking whether a requested file is present in the registry should not require scanning the entire cache directory. That is error-prone since unrelated cache files can also cause false failures; it also adds unnecessary I/O.
There was a problem hiding this comment.
check_registry() might be a misleading name. This method is meant to return all files not in the self.cache_path per the docstring
Return paths relative to :attr:`cache_path` for cache files that are
missing from the registry.
I originally made it when I was refactoring the fetch() method where it would toss error to prevent unintentional update of the registry (which would be a security issue)
if LOAD_FROM_CACHE:
registry_dictionary = self.read_registry(db_path)
missing_files_list = self.check_registry(
db_path, files=list(requested_files)
)
if len(missing_files_list) != 0:
MISSING_FILES = True
if MISSING_FILES and not APPEND_DATABASE:
raise ValueError(
"fetch() is requesting files not found in the registry. "
+ f"The missing files are {missing_files_list}. "
+ "To fix this, please set append_db=True to append the "
+ "registry."
)There was a problem hiding this comment.
This method could be renamed because it's not really checking like return a True or False
Co-authored-by: Yuxuan Zhuang <yuzhuang@stanford.edu>
Co-authored-by: Yuxuan Zhuang <yuzhuang@stanford.edu>
Co-authored-by: Yuxuan Zhuang <yuzhuang@stanford.edu>
Co-authored-by: Yuxuan Zhuang <yuzhuang@stanford.edu>
Fixes #5429 and #5431
Changes made in this Pull Request:
fetch.fetchers.StaticFetcher()fetch.pdb.from_pdb()to useStaticFetchers()LLM / AI generated code disclosure
LLMs or other AI-powered tools (beyond simple IDE use cases) were used in this contribution: yes / no
From comment: #5436 (comment)
PR Checklist
package/CHANGELOGfile updated?package/AUTHORS? (If it is not, add it!)Developers Certificate of Origin
I certify that I can submit this code contribution as described in the Developer Certificate of Origin, under the MDAnalysis LICENSE.