5,698 questions
0
votes
0
answers
17
views
How to stream from an merged layer (apply_changes) table into a downstream silver layer as stream and not Materialized View (MV)?
The Architecture: I am implementing a Delta Live Tables (DLT) pipeline following the Medallion architecture.
Landing: Auto Loader ingesting raw files (JSON/CSV).
Bronze Layer: Uses dlt.apply_changes() ...
- 309
-1
votes
1
answer
42
views
Getting unrecognized arguments: --federated-token in creating pipeline in microsoft/MLOpsPython
I am using this repo to create a mlops pipeline in Azure Devops.
When i tried to run the CI pipeline, I am getting the unrecognized Federal Token. I asked chatgpt, it says to update the cli version. I ...
2
votes
2
answers
181
views
how to get PowerShell to gracefully accept a multi line, array, system.object input to a `[string]` parameter?
ff
I have a function foo, it needs to accept a string, this string can be passed to it directly by typing at the terminal, from a variable or from the clipboard.
function foo{
[CmdletBinding()]
...
- 1,767
0
votes
1
answer
88
views
Pull the docker image in GitLab registry from pipeline
I made a Kernel project (v5.15.15) in my GitLab account and setting up build pipeline.
Did some search and made a following .gitlab-ci.yml file.
stages: # List of stages for jobs, and their ...
- 421
1
vote
1
answer
51
views
unable to find class for annotation @Field in jenkins pipeline script from scm
I'm encountering an issue with my pipeline script when running it from SCM. The @Field annotation isn't being recognized, even though I'm using the correct import statement (import groovy.transform....
0
votes
0
answers
30
views
GitLab CI/CD pipeline Salesforce code coverage test
I'm currently building a CI/CD pipeline in GitLab, and I'm running into trouble with integrating a code coverage check.
What I’ve Got So Far:
A working GitLab Runner (on Windows).
An Apex class and a ...
- 11
1
vote
1
answer
122
views
How to pull images from AWS ECR via pull through cache in .gitlab-ci
I'm using a gitlab pipeline, where its base image repo is in AWS ECR. I've configured AWS ECR pull through cache where my upstream registry is docker hub.
In my runners, I've added this
[[...
0
votes
0
answers
52
views
Unable to upload artifacts into Nexus for my Jenkins freestyle job
This is the issue that I'm still having it with Nexus (Situation: I can't deploy into nexus - I can't upload artifacts into nexus for any of my freestyle project or pipeline project in Jenkins)
By the ...
2
votes
0
answers
48
views
iOS security keychain issues in pipeline no valid identity
I am trying to use Fastlane in order to publish the app. In my pipeline script, I’m doing the following steps:
security unlock-keychain -p "$KEYCHAIN_PASSWORD" ~/Library/Keychains/login....
- 23
3
votes
2
answers
96
views
Is there a way to add more variables to the pipeline from the 'begin' section of a script?
Is there a way to dynamically add more variables to the pipeline in the "begin" section of a script?
I have a script which can accept data in three different ways:
Value as parameter
Pipe ...
- 381
0
votes
0
answers
86
views
How to add ID to streaming table in Databricks?
I want to create DLT pipeline like following. I have streaming bronze table which looks like:
Name
City
Country
John
Tokyo
Japan
Mary
New York
USA
Sam
Tokyo
Japan
Now I need two more tables. Main ...
- 1
0
votes
0
answers
35
views
PSORTb Missing output file(s) error in Nextflow process
I'm a beginner here. I've built a few nextflow workflows for other tools before. The command for PSORTb requires you to specify the directory where the output in stored and this is where I feel the ...
0
votes
1
answer
175
views
Script issue: jobs:run-tests config contains unknown keys: id_tokens, secrets, image, artifacts, coverage
Hello,
I am updating a GitLab CI/CD pipeline to add a new job.
Unfortunately, when I commit and the pipeline run, I got this error message:
jobs:run-tests config contains unknown keys: id_tokens, ...
- 131
0
votes
0
answers
34
views
Force fail build on remote agent disconnected
how can I reliably fail a Jenkins pipeline build if the remote agent disconnects during execution?
binding.setVariable('selectedNode', "")
pipeline {
agent {label: LOCAL_NODE}
...
2
votes
0
answers
101
views
Any efficient solutions to address the dependency that 'the j-th step of the i-th task is dependent on the (j+1)-th step of the (i-1)-th task'?
I have a workload consisting of a series of tasks and each task can be divided into k steps. So far, the j-th step of the i-th task depends on the j-th step of the (i-1)-th task. This can be easily ...
- 21
1
vote
1
answer
56
views
How to remove progress bar in targets package in R
In the targets package in R, does anyone know if it's possible to remove the progress when running tar_make(). I plan to save the logs as a text file but since the progress bar is being printed every ...
- 21
1
vote
1
answer
27
views
Does Boost Redis support Autopipelining
Many redis clients claim to support Auto-pielining. Does Boost redis support that? If yes, how can it be enabled?
- 41
0
votes
1
answer
69
views
How to generate unique IDs during S3-to-SingleStore columnstore ingestion (AUTO_INCREMENT not supported in SHARDed table)
I’m working on a data pipeline that ingests JSON data from an S3 bucket into a columnstore table in SingleStore.
The challenge is that I need to assign a unique ID (Primary Key) to each record during ...
1
vote
1
answer
456
views
Gitlab multi project pipeline - This job could not start because it could not retrieve the needed artifacts
So I have another project which contains artifacts I want to reuse on my project/job/pipeline. So I try to trigger the "another-project"'s pipeline to build the artifacts which I want to ...
- 2,017
0
votes
1
answer
59
views
Differential Code Coverage in Azure DevOps (ADO) for logical monorepo
I'm working on a project, where I have a pull request pipeline for a logical monorepo.
For this I'm using the git submodule functionality to pull a few repositories into mine, and building them.
I use ...
- 1,011
0
votes
0
answers
47
views
Jenkins in Docker Cannot Find NPM: npm: not found (Exit Code 127)
Description:
I am running Jenkins inside a Docker container on Windows 11. I have the Ubuntu app installed from the Microsoft Store, which already has npm version 8.5.1 installed.
However, when I run ...
0
votes
1
answer
53
views
When Merging the code, all pipeline getting triggered
I have admin api and public api. 2 seperate branch has been created and 2 different pipelines were created. When i merge the public api code, the admin pipeline also getting triggered, which should ...
0
votes
1
answer
92
views
Snakemake - Call multiple scripts in a rule
I have a snakemake rule where I call a python script with the script keyword:
rule merge_results:
input:
[...]
output:
path_results
script:
"merge_results.py&...
- 153
0
votes
3
answers
521
views
Auto-Start Self-Hosted Azure DevOps Agent After Restart
We have a self hosted agent running our builds. We have configured it to auto shutdown everyday. In the morning, we always have to go and run the pipeline agent. Is there a way we can make it run ...
0
votes
0
answers
20
views
LinearRegression object has no attr coef_ (in pipeline)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
num_features, bin_features, cat_features = split_features(X)
preprocessor = ColumnTransformer([
('num', ...
- 1