Skip to content

POC: POC branch for external file implementation using cloud auth #1462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Akshay2191 wants to merge 18 commits into
base: external-file-mgmt-integration
Choose a base branch
from
Draft

POC: POC branch for external file implementation using cloud auth #1462

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ae7fa14
third party config support
Akshay2191 Nov 12, 2025
14f1190
lint fix
Akshay2191 Nov 12, 2025
84b2310
Merge branch 'main' into external-file-mgmt-integration
Akshay2191 Nov 12, 2025
d31b2df
Added a generic file timeout
Akshay2191 Nov 13, 2025
b7a5c70
merging main to resolve conflicts
Akshay2191 Nov 13, 2025
501e08f
added generic timeout to create connection
Akshay2191 Nov 13, 2025
b7eaef7
added uT
Akshay2191 Nov 19, 2025
a83e45b
Added UT
Akshay2191 Nov 19, 2025
529c839
fix lint issues
Akshay2191 Nov 19, 2025
2a51d22
Added UT for Proxy URL
Akshay2191 Nov 19, 2025
f22834f
increasing UT coverage
Akshay2191 Nov 19, 2025
4a2b917
updated Mock management to test external file config apply
Akshay2191 Nov 20, 2025
b7759e1
PR review comments addressed
Akshay2191 Nov 24, 2025
bd60017
worked on PR comments
Akshay2191 Nov 24, 2025
38913b5
worked on PR comments
Akshay2191 Nov 28, 2025
fb3b613
removed duplicate rename function
Akshay2191 Dec 9, 2025
91061cb
POC for cloud provider auth
Akshay2191 Dec 17, 2025
1fa0c17
fix push error
Akshay2191 Dec 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/grpc/mpi/v1/command.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions api/grpc/mpi/v1/command_grpc.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/grpc/mpi/v1/common.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/grpc/mpi/v1/files.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 8 additions & 8 deletions api/grpc/mpi/v1/files_grpc.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

51 changes: 36 additions & 15 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ toolchain go1.24.10
require (
buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.9-20250912141014-52f32327d4b0.1
buf.build/go/protovalidate v1.0.0
cloud.google.com/go/storage v1.58.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.0
github.com/aws/aws-sdk-go-v2/config v1.31.15
github.com/aws/aws-sdk-go-v2/service/s3 v1.94.0
github.com/cenkalti/backoff/v4 v4.3.0
github.com/docker/docker v28.4.0+incompatible
github.com/fsnotify/fsnotify v1.9.0
Expand Down Expand Up @@ -81,24 +85,30 @@ require (
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.27.0
golang.org/x/mod v0.29.0
golang.org/x/sync v0.17.0
golang.org/x/sync v0.18.0
google.golang.org/api v0.256.0
google.golang.org/protobuf v1.36.10
)

require (
cel.dev/expr v0.24.0 // indirect
cloud.google.com/go/auth v0.16.5 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.17.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.5.3 // indirect
cloud.google.com/go/monitoring v1.24.2 // indirect
dario.cat/mergo v1.0.2 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.19.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.12.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.5.0 // indirect
github.com/DataDog/datadog-agent/pkg/obfuscate v0.73.0-devel.0.20251030121902-cd89eab046d6 // indirect
github.com/DataDog/datadog-go/v5 v5.8.1 // indirect
github.com/DataDog/go-sqllexer v0.1.9 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.29.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.54.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.54.0 // indirect
github.com/HdrHistogram/hdrhistogram-go v1.1.2 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/alecthomas/participle/v2 v2.1.4 // indirect
Expand All @@ -108,19 +118,22 @@ require (
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/apache/arrow-go/v18 v18.2.0 // indirect
github.com/apache/thrift v0.22.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.39.4 // indirect
github.com/aws/aws-sdk-go-v2/config v1.31.15 // indirect
github.com/aws/aws-sdk-go-v2 v1.41.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.18.19 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.16 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.29.8 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.38.9 // indirect
github.com/aws/smithy-go v1.23.1 // indirect
github.com/aws/smithy-go v1.24.0 // indirect
github.com/axiomhq/hyperloglog v0.2.5 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bmatcuk/doublestar/v4 v4.9.1 // indirect
Expand All @@ -129,6 +142,7 @@ require (
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect
Expand All @@ -144,12 +158,15 @@ require (
github.com/ebitengine/purego v0.9.0 // indirect
github.com/elastic/go-grok v0.3.1 // indirect
github.com/elastic/lunes v0.1.0 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.35.0 // indirect
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
github.com/expr-lang/expr v1.17.6 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/foxboron/go-tpm-keyfiles v0.0.0-20250903184740-5d135037bd4d // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/go-jose/go-jose/v4 v4.1.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
Expand All @@ -168,7 +185,7 @@ require (
github.com/google/flatbuffers v25.2.10+incompatible // indirect
github.com/google/go-tpm v0.9.6 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/grafana/clusterurl v0.2.1 // indirect
Expand Down Expand Up @@ -248,6 +265,7 @@ require (
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/prometheus/otlptranslator v1.0.0 // indirect
Expand All @@ -260,6 +278,7 @@ require (
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
github.com/splunk/stef/go/grpc v0.0.8 // indirect
github.com/splunk/stef/go/otel v0.0.8 // indirect
github.com/splunk/stef/go/pdata v0.0.8 // indirect
Expand All @@ -278,6 +297,7 @@ require (
github.com/valyala/fastjson v1.6.4 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
github.com/zeebo/errs v1.4.0 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/collector v0.139.0 // indirect
Expand Down Expand Up @@ -326,6 +346,7 @@ require (
go.opentelemetry.io/collector/service v0.139.0 // indirect
go.opentelemetry.io/collector/service/hostcapabilities v0.139.0 // indirect
go.opentelemetry.io/contrib/bridges/otelzap v0.13.0 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.36.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 // indirect
go.opentelemetry.io/contrib/otelconf v0.18.0 // indirect
Expand All @@ -352,15 +373,15 @@ require (
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/arch v0.20.0 // indirect
golang.org/x/exp v0.0.0-20251009144603-d2f985daa21b // indirect
golang.org/x/oauth2 v0.32.0 // indirect
golang.org/x/oauth2 v0.33.0 // indirect
golang.org/x/telemetry v0.0.0-20251008203120-078029d740a8 // indirect
golang.org/x/time v0.14.0 // indirect
golang.org/x/tools v0.38.0 // indirect
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
gonum.org/v1/gonum v0.16.0 // indirect
google.golang.org/api v0.250.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250929231259-57b25ae835d4 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250929231259-57b25ae835d4 // indirect
google.golang.org/genproto v0.0.0-20250922171735-9219d122eba9 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251111163417-95abcf5c77ba // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading
Loading