Skip to content

Release to Main#243

Merged
hackalog merged 42 commits intomasterfrom
dev
Feb 1, 2023
Merged

Release to Main#243
hackalog merged 42 commits intomasterfrom
dev

Conversation

@acwooding
Copy link
Copy Markdown
Collaborator

@acwooding acwooding commented Jan 10, 2023

Features

  • Add copy_dataset as a built in transformer function
  • Add new helper functions in helper.py:
    • dataset_from_fsurl: Create a dataset from the contents of an fsspec URL
    • metadata_from_fsurl: Create metadata, FILESET file list from fsspec URL
    • derived_dataset: Create a derived dataset via a single function call on .data from a
      previous dataset
  • Add support for using arbitrary conda channels in the environment.yml while forcing the order the channels getting added using a channel-order section (coming in Extend environment management to handle arbitrary conda channels #245)

API Changes

  • .DESCR -> .README
  • .EXTRA -> .FILESET

Documentation Updates

  • Update docs to be more clear and concise
  • Re-add process_20_newsgroups as a working example of a process function

Tests

  • Use a miniconda image directly instead of python image and installing miniconda on it
  • Fix docstring tests in _paths.py

@acwooding acwooding changed the title Dev Release to Main Jan 10, 2023
@acwooding acwooding added this to the 2.0 milestone Jan 30, 2023
@acwooding acwooding requested a review from hackalog February 1, 2023 15:34
@acwooding acwooding modified the milestones: 2.0, 1.6 Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants