Skip to content

[GSProcessing] Put Entry Script inside the Docker to avoid version mismatch #1330

@jalencato

Description

@jalencato

Issue we have observed because distributed_executor.py contains lots of code and sometimes includes breaking changes between versions:

  • The entry point is uploaded and attached during job launch/pipeline creation, from local graphstorm repo
  • If a user cloned the graphstorm repo from main, they get the latest version of the entry point, which in might contain a backwards incompatible change
  • When launching a sagemaker job, it will use the latest entry point, on a container that was built using e.g v0.4.2
  • As a result, it's looking for library function that was renamed, causing an error.

Ideally the entry point should be very thin, containing argument parsing and one library function call which doesn't change

Any backwards-incompatible changes to the entry point need to marked as breaking (e.g. changing the name of one of its imports)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions