Provide error when no config was found

What this PR does / why we need it:
The control loop currently segfaults when none of the config files was found. This commit makes it give an actual error, providing the user with useful feedback.
Which issue(s) this PR fixes
If needed, I'll raise an issue, please let me know.
Special notes for your reviewer (optional):

Sure, if I can bother you to look at #226 😉 (unrelated to this issue, but I think it deserves some attention too)

Could you also add a couple of unit tests to https://github.com/k8snetworkplumbingwg/whereabouts/blob/master/pkg/config/config_test.go ?
Let me know if you need any help with it.

Well I tried to, but then I discovered that a lot (maybe all) tests in config_test.go are failing now, because GetFlamIPAM (which throws an error when no config files are present) is invoked from LoadIPAMConfig. I'm not really sure if LoadIPAMConfig needs a bit of refactoring to detect this type of error, or if the tests need rewriting, or maybe even something else.
Do you have a hint for me?

Could you also add a couple of unit tests to https://github.com/k8snetworkplumbingwg/whereabouts/blob/master/pkg/config/config_test.go ?
Let me know if you need any help with it.

Well I tried to, but then I discovered that a lot (maybe all) tests in config_test.go are failing now, because GetFlamIPAM (which throws an error when no config files are present) is invoked from LoadIPAMConfig. I'm not really sure if LoadIPAMConfig needs a bit of refactoring to detect this type of error, or if the tests need rewriting, or maybe even something else.
Do you have a hint for me?

Yeah, so now that you return an error when there's no config present (I agree this should happen), the tests should be re-written to ensure this configuration is available to the tests.
Let's make sure @dougbtv is on board with the conceptual change first if you don't mind.
I can guide you on adapting the tests later on.
Stay tuned.

FWIW, the changes you need to do in the tests are quite small.
Here's what you need to do:
       var tmpDir string // this must be accessible from the tests

       BeforeEach(func() {
               var err error
               tmpDir, err = os.MkdirTemp("", "whereabouts") // must create a tmp dir to store the configs
               Expect(err).NotTo(HaveOccurred())
       })

       AfterEach(func() {
               Expect(os.RemoveAll(tmpDir)).To(Succeed()) // must clean-up the generated configs
       })

      It("can load a basic config", func() {
		conf := `` // each test will have a different conf; you don't have to change anything here

		confPath := fmt.Sprintf("%s/%s", tmpDir, "whereabouts.conf")
		Expect(os.WriteFile(confPath, []byte(conf), 0755)).To(Succeed()) // must persist the conf in the file-system

		ipamconfig, _, err := LoadIPAMConfig([]byte(conf), "", confPath) // must pass the new conf-path to the func; it only knows about the default paths
		Expect(err).NotTo(HaveOccurred())
		... // further expectations, each tests looks for different stuff
	})

@jonkerj alright! @dougbtv 's on board with this change.
Please update the failing tests in a similar way to what I'm proposing in this link.
Also please add a unit test that expects an error to be thrown - because the config on disk is not available.

Hi, sounds allright, I will look into that in a week or so.

I've updated my PR with a commit that fixes those tests. All is green on my side. I am recycling the filename whereabouts.conf throughout the tests, because (if I understand everything correct) it gets reinitialized every test (BeforeEach et al) or gets overwritten anyways.

You have plenty of failed unit tests in cmd/whereabouts_test.go .
You need to take care of them.

You are right, I need to fix the unit tests on way more places than I did. I only tested github.com/k8snetworkplumbingwg/whereabouts/pkg/config, previously, my bad.

OK, we are almost there. There are still 5 unit tests failing in cmd/whereabouts_test.go, and I'm again unsure how to proceed. In one of the tests (allocate and release addresses on ADD/DEL), ipamConfig(..) is called, which in turn calls config.LoadIPAMConfig(..), which now throws an error.
I think simply ignoring ConfigFileNotFoundError in cmd/whereabouts_tests.go around lines 1451-1454 would mimic the previous behavior (as no config found would be ignored anyways), right?
Also: I feel a bit awkward about both passing byte[](conf) and write it to a file and pass that to LoadIPAMConfig (feels like it's not really DRY). How do you think about that?

OK, we are almost there. There are still 5 unit tests failing in cmd/whereabouts_test.go, and I'm again unsure how to proceed. In one of the tests (allocate and release addresses on ADD/DEL), ipamConfig(..) is called, which in turn calls config.LoadIPAMConfig(..), which now throws an error.
I think simply ignoring ConfigFileNotFoundError in cmd/whereabouts_tests.go around lines 1451-1454 would mimic the previous behavior (as no config found would be ignored anyways), right?

Yeah, that was the previous behavior.
... but IMO, the goal of this PR is to get rid of the previous behavior and ensure an error is thrown when a config is not provided. This must be reflected in the unit tests.
IMO, you cannot just ignore the error; you should either do a proper setup of the tests,  or mock the real component, providing an implementation that does what you want.
Maybe the later is simpler.

Also: I feel a bit awkward about both passing byte[](conf) and write it to a file and pass that to LoadIPAMConfig (feels like it's not really DRY). How do you think about that?

The whole API could be improved to be honest ...
If you choose to pursue that, please do it in a separate PR.

OK, we are almost there. There are still 5 unit tests failing in cmd/whereabouts_test.go, and I'm again unsure how to proceed. In one of the tests (allocate and release addresses on ADD/DEL), ipamConfig(..) is called, which in turn calls config.LoadIPAMConfig(..), which now throws an error.
I think simply ignoring ConfigFileNotFoundError in cmd/whereabouts_tests.go around lines 1451-1454 would mimic the previous behavior (as no config found would be ignored anyways), right?

I did something else: I os.WriteFile'd the mashalled config to a tempfile, similar to your earlier suggestion. As the function prototype of ipamConfig does not allow for passing errors currently (that needs significant rewrite of the 5 involved tests), it nils out when anything fails (which cause the test to fail). Not the prettiest, but all tests pass now.

you're welcome!

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

5 of 8 (62.5%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 71.976%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/config/config.go
5
8
62.5%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
976

Relevant Lines:
1356

💛  - Coveralls


Remove the should prefix from all unit tests titles

Is your feature request related to a problem? Please describe.
A good portion of our unit tests starts with the word should which is an unnecessary filler word.
Describe the solution you'd like
Remove should from the beginning of our unit test names
Describe alternatives you've considered
N/A
Additional context

Hello,
Hope you're doing well.
I would like to work on this issue.
Thanks and Regards,
Mohit Kambli


add-detector-code

add base bad node detector code
towards https://github.com/giantswarm/giantswarm/issues/10357, build from https://github.com/giantswarm/aws-operator/pull/2546
this is the main code for a library that is used for detecting bad nodes that will be terminated
it should run in each provider-specific operator and it will return a list of nodes which should be terminated
This  function should be executed in a reconciliation loop for each cluster (some cluster CR)
There are a few configuration options:
    // MaxNodeTerminationPercentage defines a maximum percentage of nodes that will be returned as 'marked for termination'
	// ie: if the value is 0.5 and cluster have 10 nodes, than `DetectBadNodes`can only return maximum of 5 nodes
	// marked for termination at single run
	MaxNodeTerminationPercentage float64
	// NotReadyTickThreshold defines a how many times the node must bee seen as NotReady in order to return it as 'marked for termination'
	NotReadyTickThreshold int
	// PauseBetweenTermination defines a pause between 2 intervals where node termination can occur.
	// This is a safeguard to prevent nodes being terminated over and over or to not terminate too much at once.
	// ie: if the value is 5m it means once it returned nodes for termination it wont return another nodes for another 5 min.
	PauseBetweenTermination time.Duration

All have some sane defaults if not defined.
implementation of PauseBetweenTermination will be seaprate PR to have the scope of the PR small

RFR @xh3b4sd


Display categories archive for custom posts?

Eh, i make the same question time ago, but i have no response. Anyway, i just want to make an archive for the categories attached to a custom post.
I have an archive.twig and it display the regular posts (post_type=post) but no the custom posts with the same category.
I tried something like this:
$args = array(
    'post_type'   => 'custom',
    'tax_query' => array(
        array(
            'taxonomy' => 'category',
            'field'    => 'slug',
            'terms'    => 'finalizados',
        ),
    ),
);

$data['custom'] = Timber::get_posts($args);

In the archive.php and then iterate the array of "custom" in a archive-finalizados.twig... but nothing is displayed.
Any idea with the category archives, please? I really need help here, i tried and read a lot of things but nothing works for me... i'm completely stuck at this point.

Well, the solution exists, but it's not evident... all you need here is to add some things to the functions.php.
First of all, this function:
function add_custom_types_to_tax( $query ) {
    if( is_category() || is_tag() && empty( $query->query_vars['suppress_filters'] ) ) {

        // Get all your post types
        $post_types = get_post_types();

        $query->set( 'post_type', $post_types );
        return $query;
    }
}

Note: i tried to add this into the StarterSite Class, but it gives me an error "function does not exists", so i put the function outside of the class and that's all.
Then, into StarterSite Class, __contrsuct: add_filter( 'pre_get_posts', 'add_custom_types_to_tax' );
And that's all.
@jarednova it's possible to add that snippet for default in the next version of Timber?

@nanotime
The reason the function didn't work inside the Class is because
add_filter( 'pre_get_posts', 'add_custom_types_to_tax' );
needs to be:
add_filter( 'pre_get_posts', array($this, 'add_custom_types_to_tax') );
When calling a method the second argument needs to be an array, specifying what Class and method to call.
Hope that helps

Hi @nanotime - as this has been solved, I'm going to go ahead and close this.
If you want to discuss this further feel free to jump into gitter


Titancna cytoband hg19/GRCh37

Hi Brad, I see that Titancna now complain about cytoband with hg19/GRCh37 .
Thanks!
Alessandro
Error in cytoband[, "chrom"] : incorrect number of dimensions
[2018-10-26T01:17Z] lt14: Optimal state path computation: Using 4 cores.
[2018-10-26T01:17Z] ls06: Optimal state path computation: Using 4 cores.
[2018-10-26T01:18Z] lt14: Writing results to /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.titan.txt, /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.segs.txt, /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.params.txt
[2018-10-26T01:18Z] lt14: titan: Saving parameters to /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.params.txt
[2018-10-26T01:18Z] lt14: $dens.bw
[2018-10-26T01:18Z] lt14: [1] 0.6061776
[2018-10-26T01:18Z] lt14: $scat
[2018-10-26T01:18Z] lt14: [1] 0.6357545
[2018-10-26T01:18Z] lt14: $S_Dbw
[2018-10-26T01:18Z] lt14: [1] 1.241932
[2018-10-26T01:18Z] ls06: Writing results to /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.titan.txt, /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.segs.txt, /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.params.txt
[2018-10-26T01:18Z] ls06: titan: Saving parameters to /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.params.txt
[2018-10-26T01:18Z] ls06: $dens.bw
[2018-10-26T01:18Z] ls06: [1] NaN
[2018-10-26T01:18Z] ls06: $scat
[2018-10-26T01:18Z] ls06: [1] Inf
[2018-10-26T01:18Z] ls06: $S_Dbw
[2018-10-26T01:18Z] ls06: [1] NaN
[2018-10-26T01:18Z] lt14: Error in cytoband[, "chrom"] : incorrect number of dimensions
[2018-10-26T01:18Z] lt14: Calls: plotIdiogram
[2018-10-26T01:18Z] lt14: In addition: Warning messages:
[2018-10-26T01:18Z] lt14: 1: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] lt14:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] lt14: 2: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] lt14:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] lt14: 3: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] lt14:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] lt14: 4: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] lt14:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] lt14: 5: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] lt14:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] lt14: Execution halted
[2018-10-26T01:18Z] lt14: Uncaught exception occurred
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; unset R_HOME && unset R_LIBS && export PATH=/lila/data/abdelwao-lab/pastorea/bcbio/anaconda/bin:$PATH && titanCNA.R --id R_83411_16 --hetFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/R_83411_16/titancna/batch_R_83411_16-annotated-annotated-somatic-priority-vardict-prep.csv --cnFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/R_83411_16/titancna/R_83411_16-normalized.cn --numClusters 1 --ploidy 2 --numCores 4 --outDir /scratch/pastore/tmpXGtGGm --libdir None --chrs "c('1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','X')"  --genomeBuild hg19 --alphaK=2500 --alphaKHigh=2500
There were 11 warnings (use warnings() to see them)
Attaching package: 'Biostrings'
The following object is masked from 'package:base':
strsplit
Running TITAN...
titan: Loading data /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/R_83411_16/titancna/batch_R_83411_16-annotated-annotated-somatic-priority-vardict-prep.csv
titan: Loading GC content and mappability corrected log2 ratios...
Warning message:
In [<-.data.table(x, j = name, value = value) :
Supplied 376159 items to be assigned to 377322 items of column 'chr' (recycled leaving remainder of 1163 items).
titan: Extracting read depth...
Removed Chrs:
titan: Loading default parameters
titan: Using 4 cores.
titan: Parameter estimation
Optimal state path computation: Using 4 cores.
Writing results to /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.titan.txt, /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.segs.txt, /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.params.txt
titan: Saving parameters to /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.params.txt
$dens.bw
[1] 0.6061776
$scat
[1] 0.6357545
$S_Dbw
[1] 1.241932
Error in cytoband[, "chrom"] : incorrect number of dimensions
Calls: plotIdiogram
In addition: Warning messages:
1: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
2: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
3: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
4: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
5: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
Execution halted
' returned non-zero exit status 1
[2018-10-26T01:18Z] lt14: Unexpected error
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 52, in _setup_logging
yield config
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 436, in detect_sv
return ipython.zip_args(apply(structural.detect_sv, *args))
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/init.py", line 200, in detect_sv
for svdata in caller_fn(items):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/titancna.py", line 35, in run
out_dir = _run_titancna(cn_file, het_file, ploidy, num_clusters, work_dir, paired.tumor_data)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/titancna.py", line 125, in _run_titancna
do.run(cmd.format(**locals()), "TitanCNA CNV detection: ploidy %s, cluster %s" % (ploidy, num_clusters))
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; unset R_HOME && unset R_LIBS && export PATH=/lila/data/abdelwao-lab/pastorea/bcbio/anaconda/bin:$PATH && titanCNA.R --id R_83411_16 --hetFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/R_83411_16/titancna/batch_R_83411_16-annotated-annotated-somatic-priority-vardict-prep.csv --cnFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/R_83411_16/titancna/R_83411_16-normalized.cn --numClusters 1 --ploidy 2 --numCores 4 --outDir /scratch/pastore/tmpXGtGGm --libdir None --chrs "c('1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','X')"  --genomeBuild hg19 --alphaK=2500 --alphaKHigh=2500
There were 11 warnings (use warnings() to see them)
Attaching package: 'Biostrings'
The following object is masked from 'package:base':
strsplit
Running TITAN...
titan: Loading data /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/R_83411_16/titancna/batch_R_83411_16-annotated-annotated-somatic-priority-vardict-prep.csv
titan: Loading GC content and mappability corrected log2 ratios...
Warning message:
In [<-.data.table(x, j = name, value = value) :
Supplied 376159 items to be assigned to 377322 items of column 'chr' (recycled leaving remainder of 1163 items).
titan: Extracting read depth...
Removed Chrs:
titan: Loading default parameters
titan: Using 4 cores.
titan: Parameter estimation
Optimal state path computation: Using 4 cores.
Writing results to /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.titan.txt, /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.segs.txt, /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.params.txt
titan: Saving parameters to /scratch/pastore/tmpXGtGGm/R_83411_16_cluster01.params.txt
$dens.bw
[1] 0.6061776
$scat
[1] 0.6357545
$S_Dbw
[1] 1.241932
Error in cytoband[, "chrom"] : incorrect number of dimensions
Calls: plotIdiogram
In addition: Warning messages:
1: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
2: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
3: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
4: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
5: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
Execution halted
' returned non-zero exit status 1
[2018-10-26T01:18Z] ls06: Error in cytoband[, "chrom"] : incorrect number of dimensions
[2018-10-26T01:18Z] ls06: Calls: plotIdiogram
[2018-10-26T01:18Z] ls06: In addition: Warning messages:
[2018-10-26T01:18Z] ls06: 1: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] ls06:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] ls06: 2: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] ls06:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] ls06: 3: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] ls06:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] ls06: 4: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] ls06:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] ls06: 5: In .replace_seqlevels_style(x_seqlevels, value) :
[2018-10-26T01:18Z] ls06:   found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
[2018-10-26T01:18Z] ls06: Execution halted
[2018-10-26T01:18Z] ls06: Uncaught exception occurred
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; unset R_HOME && unset R_LIBS && export PATH=/lila/data/abdelwao-lab/pastorea/bcbio/anaconda/bin:$PATH && titanCNA.R --id B12626_16_I_1 --hetFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/B12626_16_I_1/titancna/batch_B12626_16_I_1-annotated-annotated-somatic-priority-vardict-prep.csv --cnFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/B12626_16_I_1/titancna/B12626_16_I_1-normalized.cn --numClusters 1 --ploidy 2 --numCores 4 --outDir /scratch/pastore/tmpAOASF7 --libdir None --chrs "c('1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','X')"  --genomeBuild hg19 --alphaK=2500 --alphaKHigh=2500
There were 11 warnings (use warnings() to see them)
Attaching package: 'Biostrings'
The following object is masked from 'package:base':
strsplit
Running TITAN...
titan: Loading data /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/B12626_16_I_1/titancna/batch_B12626_16_I_1-annotated-annotated-somatic-priority-vardict-prep.csv
titan: Loading GC content and mappability corrected log2 ratios...
Warning message:
In [<-.data.table(x, j = name, value = value) :
Supplied 189666 items to be assigned to 190163 items of column 'chr' (recycled leaving remainder of 497 items).
titan: Extracting read depth...
Removed Chrs:
titan: Loading default parameters
titan: Using 4 cores.
titan: Parameter estimation
Optimal state path computation: Using 4 cores.
Writing results to /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.titan.txt, /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.segs.txt, /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.params.txt
titan: Saving parameters to /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.params.txt
$dens.bw
[1] NaN
$scat
[1] Inf
$S_Dbw
[1] NaN
Error in cytoband[, "chrom"] : incorrect number of dimensions
Calls: plotIdiogram
In addition: Warning messages:
1: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
2: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
3: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
4: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
5: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
Execution halted
' returned non-zero exit status 1
[2018-10-26T01:18Z] ls06: Unexpected error
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 52, in _setup_logging
yield config
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 436, in detect_sv
return ipython.zip_args(apply(structural.detect_sv, *args))
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/init.py", line 200, in detect_sv
for svdata in caller_fn(items):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/titancna.py", line 35, in run
out_dir = _run_titancna(cn_file, het_file, ploidy, num_clusters, work_dir, paired.tumor_data)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/titancna.py", line 125, in _run_titancna
do.run(cmd.format(**locals()), "TitanCNA CNV detection: ploidy %s, cluster %s" % (ploidy, num_clusters))
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 23, in run
_do_run(cmd, checks, log_stdout, env=env)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 103, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; unset R_HOME && unset R_LIBS && export PATH=/lila/data/abdelwao-lab/pastorea/bcbio/anaconda/bin:$PATH && titanCNA.R --id B12626_16_I_1 --hetFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/B12626_16_I_1/titancna/batch_B12626_16_I_1-annotated-annotated-somatic-priority-vardict-prep.csv --cnFile /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/B12626_16_I_1/titancna/B12626_16_I_1-normalized.cn --numClusters 1 --ploidy 2 --numCores 4 --outDir /scratch/pastore/tmpAOASF7 --libdir None --chrs "c('1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','X')"  --genomeBuild hg19 --alphaK=2500 --alphaKHigh=2500
There were 11 warnings (use warnings() to see them)
Attaching package: 'Biostrings'
The following object is masked from 'package:base':
strsplit
Running TITAN...
titan: Loading data /lila/data/abdelwao-lab/pastorea/Project/Alternative/Alternative/work/structural/B12626_16_I_1/titancna/batch_B12626_16_I_1-annotated-annotated-somatic-priority-vardict-prep.csv
titan: Loading GC content and mappability corrected log2 ratios...
Warning message:
In [<-.data.table(x, j = name, value = value) :
Supplied 189666 items to be assigned to 190163 items of column 'chr' (recycled leaving remainder of 497 items).
titan: Extracting read depth...
Removed Chrs:
titan: Loading default parameters
titan: Using 4 cores.
titan: Parameter estimation
Optimal state path computation: Using 4 cores.
Writing results to /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.titan.txt, /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.segs.txt, /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.params.txt
titan: Saving parameters to /scratch/pastore/tmpAOASF7/B12626_16_I_1_cluster01.params.txt
$dens.bw
[1] NaN
$scat
[1] Inf
$S_Dbw
[1] NaN
Error in cytoband[, "chrom"] : incorrect number of dimensions
Calls: plotIdiogram
In addition: Warning messages:
1: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
2: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
3: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
4: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
5: In .replace_seqlevels_style(x_seqlevels, value) :
found more than one best sequence renaming map compatible with seqname style "NCBI" for this object, using the first one
Execution halted
' returned non-zero exit status 1

Alessandro;
Sorry about this issue. Are you running this with hg19 (chr1 UCSC style chromosomes) or GRCh37 (1 NCBI style chromosomes)? I'm guessing GRCh37 since the command line is missing the --genomeStyle UCSC which we'd expect for hg19.
I've tested the new version, which generalizes to support hg38, with hg19 and hg38 but not yet GRCh37 so this may be something we're missing on the command line. I'd need to loop in @gavinha to see if we're doing something wrong here. We have a titanCNA.R command line with:
--chrs "c('1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','X')" --genomeBuild hg19

Is there anything else we're missing for NCBI coordinates? Thanks for any pointers; dealing with the different naming conventions for chromosomes is frustrating so we've stuck with only chr-style for build 38 but unfortunately have both for build 37.

Thanks, we haven't had any other issues crop up because of this, so this might have been a one-off problem. Please let us know if you are still running in the problem, thank you!


Expand warning customization info

adds more details about customizing client handling of warnings, since the current call out was easy to miss.
Preview of new section at https://deploy-preview-24545--kubernetes-io-master-staging.netlify.app/blog/2020/09/03/warnings/#customize-client-handling
cc @kensipe

/approve


Activity-Filter by duration

Value proposition
As a user,
I want to filter the activities based on duration
so that i can find activities that fit my available time
Description

Acceptance criteria

[ ] Activity-Filter  should be a dropdown-menu
[ ] Activity-Filter should show a preselection of filter
[ ] No filter is selected at page load
[ ] Activities should update after a filter was selected
[ ] Activities should show the filtered activities
[ ] Activities should scroll
[ ] Activity-Filter should not scroll and always be seen and available on top

Complexity
medium
Tasks

[ ] Create branch
[ ] Setup Storybook for Activity-Filter component
[ ] Implement Activity-Filter component
[ ] write specs for Activity-Filter component
[ ] Add Activity-Filter component to App

Hey Ben,
I am wondering which options will there be available when I click on the filter button. It would be nice to have a scribble to see that. If the user can filter by duration, maybe it would be also cool to show the estimated duration in the information of each activity. 🤔
Please double-check also your acceptance criteria. Some of the points are already visible in the scribble and therefore do not need to be listed there.
Additionally, I would suggest to rephrase your value proposition e.g.:
As a user,
I want to filter the activities based on duration
so that I can find results that fit my available time

Hi Benjamin,
good job! 👍 I like this user story too, but I have two remarks:

like Andrea I also think it would be a good idea to add a time frame to each duration category for the user to know what means "short break" in minutes or hours
and is there also the possibility to remove the filter to get back to the initial state of the list (maybe by clicking the filter again)? 🤔 You should specify this in your acceptance criteria

I've updated the whole story and changed the filteroption. I named the filter in a different way. Now the user can see how much spare time he should have.

Hi Benjamin,
I think this is a good idea! 👍  I approve. ✅


Failed to remove errors in logs

If there is no check , then the bug isn't fixed

[ ] Millenaire
[ ] Chisels and Bits,
[ ] MatterOverdrive
[ ] Minecolonies
[ ] EnderIO
[ ] ChanceCube
[ ] TheBetweenlands
[ ] Terraqueous
[ ] tconstruct
[ ] immersivetech
[ ] immersiveengineering

Issue confirmed with Ice and fire and Minecraft: ( com.github.alexthe666.iceandfire.entity.tile.TileEntityMyrmexCocoon ), ( EntityDragonPart['unknown'/124, l='New World', x=0.00, y=0.00, z=0.00] as it does not exist in the list. ) ( net.minecraft.tileentity.TileEntityBed )


Add nice data access API

This changes the library the following way:

The field class now also stores sizes and offsets in the message format
The value class is overhauled to automatically cast to the right types based on the message definitions
The value class can automatically cast to the users desired types
The MessageFormat contains all information needed to parse the message
The MessageFormats are resolved recursively once the header is received

Still to do:

~Nested types are resolved correctly, but the API to access in a nested way is missing~
~Make sure all the previous tests and use cases are still supported~
~Create new tests for the added functionlity~

This allows the user to read data like this:
  // Read out some data

  // List all subscription names
  for (const auto& sub : data_container->subscriptionNames()) {
    std::cout << sub << std::endl;
  }

  // Get a particular subscription
  const auto& subscription = data_container->subscription("vehicle_status");

  // Get message format of subscription
  auto message_format = subscription->format();
  std::cout << "Message format: " << message_format->name() << std::endl;

  // List all field names
  for (const std::string &field : subscription->fieldNames()) {
    std::cout << field << std::endl;
  }

  // Get particular field
  auto nav_state_field = subscription->field("nav_state");

  // Iterate over all samples
  for (const auto &sample : *subscription) {
    // always correctly extracts the type as defined in the message definition,
    // gets cast to the value you put in int.
    // This also works for arrays and strings.
    auto nav_state = sample[nav_state_field].as<int>();
    std::cout << nav_state << std::endl;
  }

  // get a specific sample
  auto sample_12 = subscription->at(12);

  // access values by name
  auto timestamp = sample_12["timestamp"].as<uint64_t>();

  std::cout << timestamp << std::endl;

Thank you for the review! I'll work on adding more unit tests.

Also merged in main

Also added a bunch of unit testing for the added functionality


NoSuchMethodError with spring-data-jpa 2.0.0.RELEASE

@Artur- any change you'll find some time and have a look at this PR and possibly release a new version?


CLI for MAL simulator

To simplify running simulations (which is the basic use case for MAL Simulator) we should have a CLI that is easy to use.
The CLI should have an api similar to:
malsim <scenario_file> <extra options>

This is also a good way to test the scenario implementation (happening on branch #33 )

Ready for review but #33 should be merged first


What about existing WAF instances?

The Closed feedback with this title ("What about existing WAF instances?") seems unresolved.
Could someone address the central question in that feedback, or reopen it?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 55f82804-ee47-ea10-8cfc-9ad8cad0cf0e
Version Independent ID: 20be6e1c-3f35-de50-df18-2b4669497ac8
Content: Configure v2 custom rules using PowerShell - Azure Web Application Firewall
Content Source: articles/web-application-firewall/ag/configure-waf-custom-rules.md
Service: web-application-firewall
GitHub Login: @vhorne
Microsoft Alias: victorh

@buildtrue To update an existing WAF policy using PowerShell, you will need to get the existing WAF policy, update it with the information you would like to add, and then update the policy.
For example, to add a new custom rule:
#get the existing policy
$policy = Get-AzApplicationGatewayFirewallPolicy -Name $policyName -ResourceGroupName $RGname
#Add an existing rule named $rule
$policy.CustomRules.Add($rule)
#update the policy
Set-AzApplicationGatewayFirewallPolicy -InputObject $policy

I have assigned the issue to the content author to evaluate and update as appropriate.

Added this info to the article.
#please-close


Stake pool list has some suspicious sorting patterns

I will attach a screenshot to the issue. If this is a problem with cardano-wallet, please let me know and I will raise a github issue in their repo.
Daedalus 1.6.0 STN5 #13825 on Mac OS, pointing at MC4 network.
Pools from about position 98 till last one (229) seem to be primarily sorted by their unique pool id value, which seems less than ideal (even random order would be more preferred to this, imho). So pools in 98-100 range have pool ids like 00a-00f, while the last pools in 220-229 range have pool ids starting with f.
As a small aside, pool id value does not appear to be in the list of fields that can be searched on to uniquely identify a pool.

Also the color gradient makes them appear they are offline or under performing pools without having performance metrics listed there is no real way to tell how these pools perform or are ranked to any wallet user.

I would argue that the problem extends much further up the list than position 98... can someone please explain to me how "LAMBO" pool still qualifies for 10th place by any metric discussed/mentioned in any previous IOHK/CF documentation?

Seems like the math has been done and just wantonly disregarded?

TITAN made a block a in epoch 6. We are still in epoch 6. Block is shown in Daedalus, but TITAN is ranked 216 as of now. After all of the other pools, that are ranked by their poolID, that have 0 blocks created?

This is effecting Crazy Pool. I started at 164! Then jumped to 88 and then back to 256. I've just minted my first three blocks in epoch 7, have pledged 200k ADA and another 50k delegated (showing up), yet I've now slipped to 258!!! I'm behind people with 500 ADA pledge and 0 mints. If we were main net, I'd be quite distraught right now; especially with k=150 coming. Can this be given some urgency??

This may help with any troubleshooting since I experienced the same thing [CYLNX]
Changes I made/noticed going from rank 180 to 28 with 50k Pledge and blocks produced:

I initially registered my pool with less than the pledged amount in my pool wallet, then re-registered my pool. Still no rank change after creating more blocks or receiving rewards at end of Epoch.
After getting off work, I saw I was rank 28 with a small amount of rewards. Prior to that, I did not have any staking rewards

Hi there!
Few things to note:
a) Pools are ordered by non-myopic member rewards which is a metric built to be "fair" and ... well, non-myopic (i.e. long-term and non-biased towards short-term view of the system). There are several things that enter into the calculation of the non-myopic member rewards, in particular: cost per epoch & margin, pool's pledge as well as the pool's relative performance.
b) To be clear about the relative part: a pool that has produced only one block in the previous epoch may have a better rank than a pool that produced hundred blocks. A typical scenario could be that, given their respective stake, the former pool did produce all blocks it was expected to whereas the latter pool may have missed (let's say, for the sake of example) half of its slots. Yet, the first pool was only elected for one block so it couldn't have a chance to produce more! That's partly why the metric is called "non-myopic", it looks at pools as if they were just saturated (best case scenario for a pool) to see what could be the best ROI for users. In principle, if all players are rationale and delegate to the top k pools, then all players are maximizing their rewards and the system operate at full capacity.
c) There are several "edge-cases" in the ordering of pools. In particular, pools that have an exactly equal non-myopic member rewards. At the moment, if pools that have identical non-myopic reward are sorted alphabetically. There's a fix coming up in cardano-wallet (see input-output-hk/cardano-wallet#1959) so that in case of a tie, pools are sorted arbitrarily (though consistently on a same machine). This way, across many wallet clients, pools are equally favored (on one wallet, pool A may show before pool B, but the order may be different on another wallet).
d) Another "edge-case" (which is more or less linked to c)) is the case of freshly registered pools. Indeed, pool's ranking implies looking at performance... but pools are only entitled to leadership on the next epoch following their registration. Worse case, there are 2 epochs for which a pool is there and can't produce any blocks. At the moment, these pools are assigned the lowest non-myopic reward possible, which isn't fair (because then they don't a get a chance to get delegators). input-output-hk/cardano-wallet#1959 also brings an additional fix where freshly registered pools will get assigned a "default" rank based on the average of the top ranking pools so that they get a chance at launch, then, after 2 epochs, they'll get to their proper "deserved" position in the competition. There's also an upcoming fix from cardano-ledger coming soon that will make it possible to make the performance calculation a bit more "real-time" by looking at the performance on-the-fly instead of only at the end of epochs.

Thanks for the detailed write-up. Hopefully these upcoming fixes arrive before mainnet and have adequate testing done on them 🤞 🤞 🤞

I don't mean to be a squeaky wheel here, but when and where do operator pledge and a pool's saturation level come into play in the "non-myopic" rewards calculation? Instead we see a pool with 0 margin and minimum (340 ADA/epoch) fixed costs out-ranking pools that have substantially more pledge which, from a non-myopic perspective should be the opposite given that the pool with a higher pledge amount has more likelihood of producing blocks in excess of the fixed costs of operation even if they have a slightly higher margin percentage.

@KtorZ's write-up is a fantastic description of what IOHK wants the ranking to do. However, it still doesn't address the original question posted. If the math is done correctly, it should have zero correlation to the unique pool IDs. The fact that the OP can find pool ranking pattern just by using pool IDs means that the "non-myopic" reward calculation is pretty much ignored.
This is pretty much a bug in Daedalus and it will hurt all honest pool operators.

@KtorZ's write-up is a fantastic description of what IOHK wants the ranking to do. However, it still doesn't address the original question posted. If the math is done correctly, it should have zero correlation to the unique pool IDs. The fact that the OP can find pool ranking pattern just by using pool IDs means that the "non-myopic" reward calculation is pretty much ignored.
This is pretty much a bug in Daedalus and it will hurt all honest pool operators.

He answered it in his point #3 - 'At the moment, if pools that have identical non-myopic reward are sorted alphabetically.'

To follow-up on this. I will read above later today. The ranking kicked in, 2 epochs after my first mint. From that moment on, I moved with pace to 69 then 38; from 282!! Happier now, but it was quite disturbing for the first 4 epochs.

All sorting issues have been fixed long ago. Closing this ticket...


Cherry-Pick: E2E tests for new streaming library (#5932)

Fixes #5815 #5862 #5865 #5867
Cherry-pick #5932

Pull Request Test Coverage Report for Build 275063

0 of 0   changed or added relevant lines in 0 files are covered.
152 unchanged lines in 12 files lost coverage.
Overall coverage increased (+0.2%) to 78.75%

Files with Coverage Reduction
New Missed Lines
%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetNextViableTime.cs
1
90.91%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetPreviousViableTime.cs
1
90.91%

/libraries/Microsoft.Bot.Builder/Streaming/StreamingRequestHandler.cs
5
68.18%

/libraries/Microsoft.Bot.Connector.Streaming/Application/LegacyStreamingConnection.cs
6
84.78%

/libraries/Microsoft.Bot.Connector.Streaming/Application/WebSocketStreamingConnection.cs
6
82.35%

/libraries/Microsoft.Bot.Connector.Streaming/Transport/TransportHandler.cs
6
96.36%

/libraries/Microsoft.Bot.Builder/Streaming/BotFrameworkHttpAdapterBase.cs
10
43.88%

/libraries/Microsoft.Bot.Connector.Streaming/Session/StreamingSession.cs
17
91.04%

/libraries/integration/Microsoft.Bot.Builder.Integration.AspNet.Core/CloudAdapter.cs
20
72.5%

/libraries/Microsoft.Bot.Connector.Streaming/Transport/WebSocketTransport.cs
24
82.24%

Totals

Change from base Build 274618:
0.2%

Covered Lines:
24807

Relevant Lines:
31501

💛  - Coveralls


GPS: fix heading offset & baudrate param

the gps heading offset used the wrong sign
add a param for the GPS baudrate, because the Trimble MB-Two does not get detected reliably when changing the baudrate.

Thanks!


Urgent actions change

The previous name converted the ZIP file into a .exe file, making it so Windows Defender would detect it as a virus

Again, search a way to test GitHub workflows online


Spline tests fail in debug mode

running RASCALINE_BUILD_TYPE=debug tox -e all-deps gives the following failures:
FAILED python/rascaline/tests/utils/splines.py::test_fourier_space_spliner[0-1.0] - rascaline.status.RascalError: internal error (this is likely a bug, please report it): assertion failed: x >= self.parameters.start && x <= self.parameters.stop
FAILED python/rascaline/tests/utils/splines.py::test_fourier_space_spliner[0-0.0] - rascaline.status.RascalError: internal error (this is likely a bug, please report it): assertion failed: x >= self.parameters.start && x <= self.parameters.stop
FAILED python/rascaline/tests/utils/splines.py::test_fourier_space_spliner[1-1.0] - rascaline.status.RascalError: internal error (this is likely a bug, please report it): assertion failed: x >= self.parameters.start && x <= self.parameters.stop
FAILED python/rascaline/tests/utils/splines.py::test_fourier_space_spliner[1-0.0] - rascaline.status.RascalError: internal error (this is likely a bug, please report it): assertion failed: x >= self.parameters.start && x <= self.parameters.stop

This looks like we are trying to evaluate the spline outside of its definition domain. @PicoCentauri would you have time to look at it?

Yes sure. I will look at on Monday.


Feature/Motorhead Memory

Issue
#368
Note
https://github.com/FlowiseAI/Flowise/issues/368#issuecomment-1623036178

Tested locally works fine for me:

and when retrieve the message:

run yarn lint-fix to fix the linting issues

Node screenshot:


Fix a JS error on mkdocs: mkdocs_page_input_path

The mkdocs_page_input_path variable is only defined on the RTD theme. For other themes, the variable is not defined.
Here's a page that manifests the error: https://crafttweaker.readthedocs.io/en/latest/

Replaces: #3536

I merged https://github.com/rtfd/readthedocs.org/pull/3536 instead of this one.


[Wallet] Add loading indicator to WebViewScreen

Description
While loading, the URL on the WebViewScreen the screen used to stay blank. Now it shows a loading indicator.
Tested
Yes, manually.
Related issues

Fixes #6462

Backwards compatibility
N/A

Hi @gnardini @jeanregisser We have verified this on latset android internal build V1.9.2 & testflight build V1.9.2 and observe the followings.
In iOS: Loading indicator is shown when user redirecting to celo.org page after tapping on  “Hey Ankit...celo.org” push notification.
in Android: Blank screen is shown instead of loading indicator when user redirecting to celo.org page after tapping on  “Hey Ankit...celo.org” push

Hi @gnardini @jeanregisser We have verified this on latset android internal build V1.9.2 & testflight build V1.9.2 and observe the followings.
In iOS: Loading indicator is shown when user redirecting to celo.org page after tapping on  “Hey Ankit...celo.org” push notification.
in Android: Blank screen is shown instead of loading indicator when user redirecting to celo.org page after tapping on  “Hey Ankit...celo.org” push


Support vmss in kubernetes deployment

Is this a request for help?:

Is this an ISSUE or FEATURE REQUEST? (choose one):
Feature
What version of acs-engine?:

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
What happened:
Kubernetes currently only supports availability set in agent pool.
What you expected to happen:
Support vmss in kubernetes agent pool.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know:

@feiskyer

vmss support will be added in kubernetes v1.10. To support vmss, acs-engine should ensure

master nodes are still running in AvailabilitySet
agent nodes are running in ScaleSet

And /etc/kubernetes/azure.json should be changed for both master and agent nodes:
{
  ...
  "vmType":"vmss",
  // remove primaryAvailabilitySetName
  "primaryScaleSetName":"<vmss-name>"
}

What is weird is that the kubernetes/autoscaler documentation references VMSS enabled kubernetes clusters... am I missing something or is this the edge or are they referring to some sort of managed cluster that runs vmss 😜

@feiskyer Could I check if it is going to support cross-zone ScaletSets when you mentioned "agent nodes are running in ScaleSet"

Could I check if it is going to support cross-zone ScaletSets when you mentioned "agent nodes are running in ScaleSet"

@yfernando It should support this, but I haven't validated it yet.

I'm a bit confused, and sorry if this is a silly question, but are there changes that need to happen in ACS-Engine to support VMSS, or is this simply waiting for Kubernetes 1.10?
I want to run the Kubernetes autoscaler on a VMSS cluster created ACS-Engine and I'm trying to figure out what configuration I need to set aside from targeting 1.10

Ah okay. Nevermind. I tried to target VMSS but got:
INFO[0000] Error returned by LoadContainerService: VirtualMachineScaleSets are not supported with Kubernetes since Kubernetes requires the ability to attach/detach disks.  To fix specify "AvailabilityProfile":"AvailabilitySet"

@forrestcoward We are waiting both. @JiangtianLi will work on acs-engine part and kubernetes v1.10 will be released in next few days.


Dereference symlinks when packaging

As a user of metaflow, I would like to share external libraries among flows in different folders without code duplication.
symlinks provide a solution but not when packaging and running --with batch, because the TarFile are keeping the symlinks as such instead of dereferencing when packaging.

@jens-hummelshoej-tri Any reasons for 356 and 357 to be different PRs?

@savingoyal no, I guess they could be the same PR. Two different approaches to the same issue. PR356 works for linked files, PR357 works for linked folders. Honestly, I thought when some tests failed on PR356 that it would be ignored, so I found a different solution.

@jens-hummelshoej-tri I am happy to let you know that symlinks are finally being followed in the latest release. We just added a bit extra logic to avoid cyclical links which could make it hang otherwise


Adds tolerations to cassandra statefulset spec

Signed-off-by: Tom Gummery<EMAIL_ADDRESS>@KongZ @maver1ck
What this PR does / why we need it:
Adds tolerations section to the pod spec so that Cassandra pods can be scheduled to tainted nodes.
Checklist

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md

/lgtm

/lgtm

/ok-to-test

/retest

/lgtm cancel

/lgtm

@u2g-tg
Thank you for the contribution.


Breakout Board

I had asked the question before, but I probably posed the question wrong.
I am trying to source the 8 channel PWM breakout board photographed in the wiki.
I have attached a screenshot of the component I am referring to.
This one seems to have a 10-pin JST input. I am unable to find a manufacturer, part number or anywhere that supplies this particular unit.
If anyone could give me any advice, guidance or a pointer I'd be very grateful. It is literally the last component I am waiting on, I have everything else ready and at hand.
Many Thanks!

Sorry, I misunderstood your question the first time, for some reason I thought the question was about receiver/transmitter.
As for the breakout board: as I remember, we ordered parts directly from TBS as an ARF kit with few customizations that included replacing DJI Naza with PixFalcon (unfortunately, TBS does not sell these kits anymore). The breakout board came with the kit. As I understand, the breakout board is actually a part of Pixhawk Mini not PixFalcon but apparently TBS included it as well. If you don't have it then you can use PWM breakout cable that comes with PixFalcon. The board is needed only when you need to power PWM rail which you don't need to do in case of the drone.
Hope this helps.

Thanks very much! You're time and effort really helped.


Dart FFI calls get invalid error

I found that binding the Rust library through Dart would cause errors when there was reqwest API call in the implementation, and I would not get a valid error kind.
This is an FFI (C ABI):
#[no_mangle]
pub extern "C" fn foo() {
    use reqwest::blocking::get;

    let resp = get("https://github.com/").unwrap();
    println!("status: {}", resp.status());
}

This is the Dart binding and calling code:
import 'dart:ffi';
import 'package:ffi/ffi.dart';

final dylib = DynamicLibrary.open('libraries/libfoo.so');

typedef foo_func = Void Function();
typedef Foo = void Function();
final Foo foo =
    dylib.lookup<NativeFunction<foo_func>>('foo').asFunction(); // Call this function

Output:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Builder, source: Normal(ErrorStack([])) }', src/libcore/result.rs:1188:5
stack backtrace:
   0:     0x7f9bebf4cea4 - backtrace::backtrace::libunwind::trace::heb43798aede8bd30
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1:     0x7f9bebf4cea4 - backtrace::backtrace::trace_unsynchronized::had2ba7dec4bd2732
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2:     0x7f9bebf4cea4 - std::sys_common::backtrace::_print_fmt::hda61f46e822731b2
                               at src/libstd/sys_common/backtrace.rs:84
   3:     0x7f9bebf4cea4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hfe37fa5de6572965
                               at src/libstd/sys_common/backtrace.rs:61
   4:     0x7f9bebf6e2dc - core::fmt::write::h74887d18db27282c
                               at src/libcore/fmt/mod.rs:1025
   5:     0x7f9bebf48d57 - std::io::Write::write_fmt::h6808f3d5eceed5e5
                               at src/libstd/io/mod.rs:1426
   6:     0x7f9bebf4ee2e - std::sys_common::backtrace::_print::hcc0fd4b3552039ef
                               at src/libstd/sys_common/backtrace.rs:65
   7:     0x7f9bebf4ee2e - std::sys_common::backtrace::print::h1c9c5c1c0505592d
                               at src/libstd/sys_common/backtrace.rs:50
   8:     0x7f9bebf4ee2e - std::panicking::default_hook::{{closure}}::hefb6085c1ab83a59
                               at src/libstd/panicking.rs:193
   9:     0x7f9bebf4eb21 - std::panicking::default_hook::h1b037d2bf0657ab3
                               at src/libstd/panicking.rs:210
  10:     0x7f9bebf4f4eb - std::panicking::rust_panic_with_hook::h787d7f532b084b9a
                               at src/libstd/panicking.rs:471
  11:     0x7f9bebf4f09e - rust_begin_unwind
                               at src/libstd/panicking.rs:375
  12:     0x7f9bebf6a19e - core::panicking::panic_fmt::h76b979c035808e69
                               at src/libcore/panicking.rs:84
  13:     0x7f9bebf6a297 - core::result::unwrap_failed::hca6a012bfa3eb903
                               at src/libcore/result.rs:1188
  14:     0x7f9bebae6f55 - foo
  15:     0x7f9bf8b03aee - <unknown>

I can't understand why this is. Do you have any clues?

Hm, I've never used Dart FFI before, so this will be new for me. The error Normal(ErrorStack([])) looks very unhelpful. Is that typical in Dart?

I have never encountered such a situation. No problem with other HTTP client libraries, such as attohttpc or http_req.
I noticed that the error may be that the request was not sent during the construction of the Request. But I can't locate the specific line of code.

Could you separate the code slightly, so we know if the error is from building the client or the request?
let client = reqwest::blocking::Client::new();
let resp = client.get("https://github.com/").send().unwrap();

If the error is from building the client, here's some places that can error: spawning a runtime thread (std::thread::spawn), creating an IO poller (like epoll_create), creating the TLS context... What reqwest features do you have enabled?

Thanks for the hint, an error occurred when calling the Client::new() function, this is the output:
thread '<unnamed>' panicked at 'Client::new(): reqwest::Error { kind: Builder, source: Normal(ErrorStack([])) }', src/libcore/result.rs:1188:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:84
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1025
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:193
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:471
  11: rust_begin_unwind
             at src/libstd/panicking.rs:375
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:84
  13: core::result::unwrap_failed
             at src/libcore/result.rs:1188
  14: reqwest::blocking::client::Client::new
  15: foo
  16: <unknown>

I have not enabled any features and have tried to enable some features, but nothing has changed.

The simplest other thing we can test for: does thread::spawn work?
If you make function like this:
#[no_mangle]
pub extern "C" fn foo_thread() {
    use std::thread

    let t = thread::spawn(|| {
        println!("hello from spawned thread");
    }).unwrap();

    t.join().unwrap();
    println!("hello after thread finished");
}

This code works fine.

Hm, my next guess then is that there is an error building the TLS context (could be failing to find openssl). What if you disable TLS from reqwest?
reqwest = { version = "0.10", default-features = false, featues = ["blocking"] }

I tested it and it failed. The error has not changed.

Sorry to have you try these random things :(
My last guess at the moment is trying to build the runtime. How about a function like this:
#[no_mangle]
pub extern "C" fn foo_rt() {
    use tokio::runtime::Runtime;

    let mut rt = Runtime::new().unwrap();

    rt.block_on(async {
        pritnln("hello from runtime");
    });
}

This code also works fine. I still hope that this problem can be solved. Reqwest is almost the only full-featured and mature HTTP client. It is a pity that it cannot be used in Flutter applications (Android/IOS).

Ah, since you mentioned Android, I think I've located that error (Normal(ErrorStack([]))): https://github.com/sfackler/rust-native-tls/blob/fb8903a07179a04260f624e0cab090aa54446fa3/src/imp/openssl.rs#L118

But I enable rustls-tls (or rustls) feature:
reqwest = {version = "0.10", default-features = false, features = ["rustls-tls"]}

The error remains unchanged.
The two libraries attohttpc and http_req that I mentioned at the beginning also use rustls, and all functions are normal.

I know others have gotten openssl to work on Android. That said, I don't believe this is an issue with reqwest. I believe it might have to do with needing to set the --target or equivalent when compiling.


Build and packaging fixes

Hi,
Please pull from build-system2 branch to get the following fixes:

Merge spec files into a single file.
Packaging is done using a single spec file, and libz, tools and vpd are created as subpackages.
Remove old specfiles.
Fixup version information in the build system.
Build system no longer fails to retrieve version information if we are building from a tar.gz.

Gabriel, thanks for providing the reworks. Unfortunately git complains about conflicts that must be resolved before we can merge the changes. Would you mind rebasing your tree and sending a conflict free version of your additions, rather than me trying to do this and potentially messing it up? Thanks.

Hi, yeah, sorry.  The conflict is with commit e574f7 from yesterday.  I fixed it, but I'm not sure how to add it to this pull request in github, so I am closing this and opening a new one...


[ALLUXIO-3294] Remove lineage

https://alluxio.atlassian.net/browse/ALLUXIO-3294
After this change, the only lineage-related code is
the handling for lineage journal entries. This is so
that users can upgrade from a version where they
tried out lineage.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/169/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/170/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/175/
Test PASSed.

remove Lineage doc?

@apc999 Which docs are you thinking of?

yay!


Const Char

Although the concat! macro seems to work fine on char literals (concat!('h', 'e', 'l', 'l', 'o')), it won't seem to accept a const char.
Example:
use constcat::concat;

const MARK: char = '?';
const QUESTION: &'static str = concat!("Why doesn't this work", MARK);

provides the error: no method named 'as_bytes' found for type 'char' in the current scope.
Is there any chance to solve this?

This would be cool to have but I don't think it is a simple fix 🤔. You will have to use const MARK: &'static str = "?" for now.
Some issues:

We would need to convert a char in a const context to a byte slice (&[u8]). Static strings can be trivially converted using .as_bytes(), but char is a bit more complex. It seems like you can do it like this, which does not work in a const context yet. 🤯 not sure if there is a better way?

const c: char = '?';

let mut buf = [0u8; 4];
c.encode_utf8(&mut buf);
let as_bytes = &buf.as_slice()[..c.len_utf8()];

The macro needs to be able to handle both types of values. Usually the way to do this is with a trait, but again this has to be done in a const context and const trait methods are not yet supported in Rust.


The current stable nodejs version is not supported

I try to reuse websocket under the most stable nodejs, but the internal error is that node deleted some methods

I would say that this module is not really supported anymore. If you’d like to send aPR to update it, I’ll be very happy to review.


Load Environment Variables when setting up an "https://" Playbooks Repo

Is your feature request related to a problem?
The Remote Playbooks feature looks great, but it has some debilitating limitations. The allowed repo URL prefixes do not allow me to clone a remote repo from the software that my org is using.
I have a playbooks repo set up in Gerrit, and anonymous cloning over HTTP is disabled. That is, all repos are private by default. Cloning over SSH requires that I use the prefix "ssh://${username}@" and "git@" is not permitted.
I would attempt to clone using the "https://" prefix instead, since it is possible (though not necessarily good practice) to clone using "https://${username}:${password}@repo". I actually tried this by adding some "additional_env_vars" to my helm values at installation time, like so:
cat << EOF >> /mnt/generated_values.yaml
runner:
  additional_env_vars:
  - name: GERRIT_USER
    valueFrom:
      secretKeyRef:
        name: robusta-gerrit
        key: user
  - name: GERRIT_PASSWORD
    valueFrom:
      secretKeyRef:
        name: robusta-gerrit
        key: password
playbookRepos:
  my_extra_playbooks:
    url: "https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo"
EOF

I do see the environment variables show up when the runner starts, and the runner does try to use this URL, but the variables are not evaluated. This doesn't work because (I think) the app doesn't load environment variables before it tries to initialize the repo. My review of the code makes it look like the environment is only loaded after the repo is initialized.
So, in essence, my org's default Git management solution is hard excluded. This is not ideal.
Describe the solution you'd like
SOLUTION ONE:
Github allows you to authenticate to private repos using personal access tokens, which can be specified in the cloning URL. https://docs.github.com/en/rest/overview/other-authentication-methods?apiVersion=2022-11-28
So does Gitlab: https://stackoverflow.com/questions/25409700/using-gitlab-token-to-clone-without-authentication
And Gerrit allows me to clone with a username and password in the URL as well.
It seems to me that it would be a short lift and a big feature boost for private repos over HTTPS if we could load environments variables before initializing the playbook repository.
Prior to install time, we could put the appropriate credentials into a kubernetes secret, and at install time we could add the appropriate configurations (as noted above) to the values that would load the environment into the runner and then have the runner try to clone from there.
SOLUTION TWO:
Add "ssh://" to the list of legal repo prefixes. This will probably be easier to implement than Solution One, though it keeps the existing limitations on what https based git URLs can do.
Describe alternatives you've considered
The easiest thing for me to do is to set up a private Playbooks repo in github after all and then go from there.
On principle, though, it seems more favorable to me to enable this functionality for as wide a set of software as possible as long as it is not too heavy a lift.
Additional context
N/A

Thanks for reporting it @prannonpendragas
In your example you addded:
"https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo"

The Robusta env var replacement syntax is: {{ env.SOME_VAR_NAME }}
Can you try:
"https://${env.GERRIT_USER}:${env.GERRIT_PASSWORD}@repo"

Thanks for reporting it @prannonpendragas In your example you addded:
"https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo"

The Robusta env var replacement syntax is: {{ env.SOME_VAR_NAME }} Can you try:
"https://${env.GERRIT_USER}:${env.GERRIT_PASSWORD}@repo"

So, I tried this out. This did not work. Oddly enough, I got different errors this time than I did the last time that I tried this.
I first tried what you suggested with the configuration:
    url: "https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo"

I received this error:
2023-03-02 15:11:33.225 INFO     Cloning git repo https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo. repo name repo
2023-03-02 15:11:34.220 ERROR    running command ['git', 'clone', 'https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo', '/app/robusta-git/repo'] failed with returncode=128
2023-03-02 15:11:34.221 ERROR    stdout=
2023-03-02 15:11:34.221 ERROR    stderr=Cloning into '/app/robusta-git/repo'...
remote: Unauthorized
fatal: Authentication failed for 'https://repo/'

2023-03-02 15:11:34.221 ERROR    Failed to add playbooks repo my_extra_playbooks
Traceback (most recent call last):
  File "/app/src/robusta/runner/config_loader.py", line 111, in __load_playbooks_repos
    repo = GitRepo(
  File "/app/src/robusta/integrations/git/git_repo.py", line 70, in __init__
    self.init_repo()
  File "/app/src/robusta/integrations/git/git_repo.py", line 133, in init_repo
    self.__exec_git_cmd(["git", "clone", self.git_repo_url, self.repo_local_path])
  File "/app/src/robusta/integrations/git/git_repo.py", line 121, in __exec_git_cmd
    raise Exception(f"Error running git command: {cmd}")
Exception: Error running git command: ['git', 'clone', 'https://${GERRIT_USER}:${GERRIT_PASSWORD}@repo', '/app/robusta-git/repo']

I then tried it using this configuration:
    url: "https://$GERRIT_USER:$GERRIT_PASSWORD@repo"

I got essentially the same error, just with different literal strings.
I then tried it using this configuration:
    url: "https://{{ env.GERRIT_USER }}:{{ env.GERRIT_PASSWORD }}@repo"

Same error.
Previously, when I had been trying this out, I got errors indicating that the environment variables could not be found, or I saw that the variables in the URL string were being evaluated to empty strings, so these new errors are a bit weird to me.
I have a feeling that maybe since I am populating my generated_values.yml file with this configuration using cat << 'EOF' >> /mnt/generated_values.yml rather than editing the values file directly, it might be causing some sort of weird behavior? But no idea.
In any case, the behavior from robusta is not ideal. What are your further thoughts?

Thank you @prannonpendragas
I'll add the support for:
   url: "https://{{ env.GERRIT_USER }}:{{ env.GERRIT_PASSWORD }}@repo"

Great news @arikalon1 ! Once you have added the support, I am happy to test it in my environment.

@prannonpendragas
Would this be sufficient for you:
url: "{{ env.REPO_URL_WITH_CREDS }}"

Meaning, that in your secret you'll have:
url: https://user:pass@repo

That might work, yeah. What would the implementation look like precisely? @arikalon1

In the generated_values.yaml:
runner:
  additional_env_vars:
  - name: USER_REPO_URL
    valueFrom:
      secretKeyRef:
        name: robusta-gerrit
        key: url
playbookRepos:
  my_extra_playbooks:
    url: "{{ env.USER_REPO_URL}}"

Inside the robusta-gerrit secret, the url key should have:
"https://user:password@repo"
Will that work?

I'll try this method tomorrow @arikalon1 , but on first review I have doubts that this will work, since in previous examples the whole problem is that the environment variable is not evaluated when Robusta tries to initialize the Git repo.
I'll let you know how it goes, though. :)

@prannonpendragas I wasn't suggesting it's working at the moment. It's not. I was asking, if this kind of fix will satisfy your requirements

Ah! Please forgive my misunderstanding.
Yes, that would satisfy my requirement. It's simple in the configuration file and fits my use case nicely. :)

Added the option to pull the playbookRepo url from an environment variable
For example:
playbookRepos:
  chatgpt_robusta_actions:
    url: "{{ env.REPO_URL }}"
runner:
  additional_env_vars:
    - name: REPO_URL
      value: "https://github.com/robusta-dev/kubernetes-chatgpt-bot.git"

It will be available on the next release (0.10.13)
The changes are available in this image if you'd like to test it:
us-central1-docker.pkg.dev/genuine-flight-317411/devel/robusta-runner:load_repo_url_from_env
To use it, add to your values:
runner:
  image: us-central1-docker.pkg.dev/genuine-flight-317411/devel/robusta-runner:load_repo_url_from_env


iOS controls appear over plyr control when exiting native fullscreen by swiping down to exit

Expected behaviour
iOS controls should never be shown and should not block plyr custom controls.
Actual behaviour
iOS controls appear over plyr control when exiting native fullscreen by swiping down the player.
This will not happen if the native player is dismissed by using the close button in the upper-left.
Steps to reproduce

Confirgure Plyr to use iOS native fullscreen
Start video playback
Swipe down to close the video player (Don't press the close button in the top left)
iOS controls will show over the Plyr control on the video preview

Environment

Browser: Safari
Version: 14
Operating System: iOS
Version: 14.7

Example swiping down to exit

Example using the close button

Same issue here, iOS controls take over in fullscreen on iOS. Note this doesn't happen on iPadOS.

@mannyveloso I don't think your issue is related. My issue is after exiting Fullscreen by swiping down. Not while in fullscreen.

 That's interesting, we see the exact opposite: when we play a video the iOS controls take over and when we swipe down we see the plyr controls.
What iOS are you on? Ours is on 14.7.1

@mannyveloso Did you look at the gif or...? When in fullscreen, iOS controls are shown because I have fullscreen: { iosNative: true }.

I did; the 2nd video shows iOS video controls before the close button is clicked and what I presume are your custom controls when the video stops.

Can you create you own issue instead of polluting mine ? Clearly your problem is totally unrelated. The fact that iOS control display in fullscreen is totally normal as I said because I have the option { fullscreen: iosNative: true }. Maybe you should go read the documentation:

Yes I will to be stopping polluting your issue. Thanks for your help and understanding, I'm sure that I'll try and do the same sometime in the future.


Make cloup pass mypy --strict

@alexreg can you please check now?

@janluke Same problem, I'm afraid.

@alexreg Are you sure you are running mypy against this branch of Cloup? I've just forked your repo and run mypy --strict against master. I only got 3 errors related to your code (not Cloup) that I fixed in a few seconds.
(venv) PS H:\Repo\_forks\alexreg-typer> mypy --version
mypy 0.960 (compiled: yes)
(venv) PS H:\Repo\_forks\alexreg-typer> mypy --strict typer
Success: no issues found in 12 source files

I've not checked if you added type: ignore everywhere though.

Yep, definitely running it against your branch. I'll experiment tomorrow if I can.

@janluke I can confirm exactly the same errors are occurring on my branch still, using your latest version of cloup. Maybe for you it passes because you have strict = False in your mypy.ini file? Not sure.
As for type: ignore, I'm not quite sure what you mean. The whole point of type hints is that you should not have to add these, in theory.

@janluke I can confirm exactly the same errors are occurring on my branch still, using your latest version of cloup.

You mean this branch, right? Of course master will give errors.

As for type: ignore, I'm not quite sure what you mean. The whole point of type hints is that you should not have to add these, in theory.

I was just wondering if I maybe didn't see any error because you added type: ignore. That's not the case.

Maybe for you it passes because you have strict = False in your mypy.ini file? Not sure.

Even if that was the case (and it's not), the CLI argument overrides settings. As you can see, I run mypy passing --strict and can't replicate the errors you're seeing.

@janluke Yeah, this branch. Re the ini file, I agree, I just thought maybe its behaviour was weird...
I deleted my mypy.ini file, and get a few more telling errors.
typer/models.py:18: error: Cannot find implementation or library stub for module named "cloup"
typer/models.py:22: error: Cannot find implementation or library stub for module named "cloup.constraints"
typer/params.py:4: error: Cannot find implementation or library stub for module named "cloup"
typer/params.py:5: error: Cannot find implementation or library stub for module named "cloup.constraints"
typer/core.py:23: error: Cannot find implementation or library stub for module named "cloup"
typer/main.py:24: error: Cannot find implementation or library stub for module named "cloup"
typer/main.py:49: error: Cannot find implementation or library stub for module named "cloup.constraints"

Every file you see in the repo (and so py.typed) is included in the distribution because I'm using setuptools-scm. Since I cannot reproduce, there's something wrong with either my or your environment, or it may depend on the version of mypy. Can you run the following?
pip install -U --force-reinstall git+https://github.com/janluke/cloup.git@strict-typing#egg=cloup
pip install -U mypy
mypy --strict typer

@janluke Agh, I see what was happening now! Homebrew can be a bit annoying sometimes... it was giving me the Python 3.9 interpreter but the Python 3.10 version of mypy. Sorry for wasting your time. It now works fine for me. I appreciate you adding this anyway... feel free to merge, of course.

No worries :)


add ci_artifacts to gitignore

@jGaboardi shall we add the ci_artifacts folder to gitignore? When running tests locally, it is created and git sees the files. Or is it going to break that upload action?

Let's add it do gitignore and remove ci_artifacts (seems we accidentally added that yesterday.
That shouldn't break the upload action, but if it does we can figure out a better solution.


Individual files for the lists and cards

When parsing the file, store the cards as individual directories, with a file for the title and body, and individual files for the attachments

I thought the attachment URLs were publicly accessible, but they aren't. If we can't programmatically authenticate, then build a webpage that opens in the browser and uses the browser cookie to download the attachments, maybe with javascript.
Look at this:
// https://plainenglish.io/blog/how-to-download-files-with-javascript
function downloadFile(url, fileName) {
  fetch(url, { method: "get", mode: "no-cors", referrerPolicy: "no-referrer" })
    .then((res) => res.blob())
    .then((res) => {
      const aElement = document.createElement("a");
      aElement.setAttribute("download", fileName);
      const href = URL.createObjectURL(res);
      aElement.href = href;
      aElement.setAttribute("target", "_blank");
      aElement.click();
      URL.revokeObjectURL(href);
    });
}


TypeError: AsymmetricAttention.init() got an unexpected keyword argument 'clip_feat_dim'

Hi! Thanks for trying out the inference repo. Can you provide the dit-config.yaml you used to run the model?

Hi, you can simple fix the error by deleting the unexpected keyworks from the dit-config.yaml. For me, I have deleted the clip_feat_dim, pooled_caption_mlp_bias and use_transformer_engine from dit-config.yaml and everything works fine. I guess its a mismatch between the ckpt on huggingface and the codebase.

If you pull the latest version of this repository, this error should be gone @cene555.


Expose the sqlite3_update_hook callback

Useful for getting data change notifications on write.
Checklist

[x] I grant to recipients of this Project distribution a perpetual,
non-exclusive, royalty-free, irrevocable copyright license to reproduce, prepare
derivative works of, publicly display, sublicense, and distribute this
Contribution and such derivative works.
[x] I certify that I am legally entitled to grant this license, and that this
Contribution contains no content requiring a license from any third party.

@tantaman can you resolve the conflicts

What was the reason this was closed for? I think this would be a really welcome addition. Is there any way someone could potentially take lead of this instead and perhaps look at getting it merged? thanks a lot :)

I don't know why @tantaman closed it, but I'm making significant internal changes for a new release, and callback handling is one of the things affected. I probably won't be merging much of anything except important bug fixes (particularly new callbacks) until the new release is ready.

Got it, thanks! Would these changes allow for triggers? In a similar fashion to this
https://github.com/WiseLibs/better-sqlite3/issues/62#issuecomment-325797335

@alii Unless I'm missing something, the comment you link describes a technique that uses sqlite3_create_function instead of sqlite3_update_hook. wa-sqlite already provides a wrapper for that.

Is there any interest in re-opening this PR? sqlite3_update_hook would enable many reactive / local first app use cases.
For example, the implementation of useLiveQuery in expo-sqlite uses the update hook:
https://github.com/expo/expo/blob/75172f0339d707a685c60e72e70a7ce4e90c9080/packages/expo-sqlite/ios/SQLiteModule.swift#L433-L457


Paramatizing region

Just make region a parameter and updated bootstrap. I thought about adding additional vars for future HA deployment for controller and nodes across regions.

Closed as discussed :)


Fix handling of spaces in getVersionFomJar

See #27.
Note that this commit is untested.

Thanks for the review, @peq. I agree completely. I've pushed the changes.

you duplicated the proc, you need to remove the upper line to make the build succeed.
Thanks anyways for the PR, instead of merging into master I will most likely just integrate it into the next-gen branch which will be merged soon.

you duplicated the proc variable, you need to remove the line above to make the build succeed.

Would have been easier if I had just let you guys fix it in the first place. :) Anyways, I've removed the duplicate line. I don't know how that happened. Mental blunder.

Thanks anyways for the PR, instead of merging into master I will most likely just integrate it into the next-gen branch which will be merged soon.

That sounds good with me. I just want #27 fixed.

Codecov Report

Merging #28 into master will not change coverage.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master      #28   +/-   ##
=========================================
  Coverage     25.91%   25.91%
  Complexity       74       74
=========================================
  Files            26       26
  Lines          1092     1092
  Branches        137      137
=========================================
  Hits            283      283
  Misses          748      748
  Partials         61       61

Impacted Files
Coverage Δ
Complexity Δ

src/main/kotlin/global/InstallationManager.kt
61.7% <100%> (ø)
15 <1> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5ac409...a6464c3. Read the comment docs.

Thanks again @frederikaalund 🍰
merged into #26


Refer to RSE-Sheffield/uos-varnish for notes on applying varnish locally

See https://github.com/RSE-Sheffield/uos-varnish

Resolved by #11


Update Ordinal Insiders

Thank you for listing us. Please update our collection to reflect the new changes. We appreciate all of your help.

ayeeeee thank you man!!!!
On Wed, May 3, 2023 at 11:12 AM MillHaus33 @.***> wrote:

Merged #2027
https://github.com/ordinals-wallet/ordinals-collections/pull/2027 into
main.
—
Reply to this email directly, view it on GitHub
https://github.com/ordinals-wallet/ordinals-collections/pull/2027#event-9156757018,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A6ZKFFQ2JICGH75EONGKWFLXEJYXRANCNFSM6AAAAAAW7I6GMA
.
You are receiving this because you authored the thread.Message ID:
<ordinals-wallet/ordinals-collections/pull/2027/issue_event/9156757018@
github.com>

We will also be releasing our very own BRC-20 soon!
I will make sure to create a push request and update you here sir!
On Wed, May 3, 2023 at 7:03 PM The DeepFunds @.***> wrote:

ayeeeee thank you man!!!!
On Wed, May 3, 2023 at 11:12 AM MillHaus33 @.***>
wrote:

Merged #2027
https://github.com/ordinals-wallet/ordinals-collections/pull/2027 into
main.
—
Reply to this email directly, view it on GitHub
https://github.com/ordinals-wallet/ordinals-collections/pull/2027#event-9156757018,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A6ZKFFQ2JICGH75EONGKWFLXEJYXRANCNFSM6AAAAAAW7I6GMA
.
You are receiving this because you authored the thread.Message ID:
<ordinals-wallet/ordinals-collections/pull/2027/issue_event/9156757018@
github.com>


Duplicated and invalid local repositories after building with vsxmake when using anti-slashes

Xmake Version
v2.6.9+HEAD.22ecc0156 (dev)
Operating System Version and Architecture
Windows 10 version 1909
Describe Bug
The local repos are messed up after I try to build with vsxmake2022 and the configuration is lost.
Expected Behavior
The local repos should be left untouched as well as the configuration.
Project Configuration
I have two local repos:
add_repositories("local-repo xmake-packages")
add_repositories("iconeus-repo https://github.com/Iconeus/xmake-repo.git")

The first one is cloned from SVN and contains pre-build binaries, and the second is a private package repository.
Additional Information and Error Logs
TL;DR: vsxmake does not seem to like antislashes at all in options

This is the initial state, the repos are correct, the project is configured, it builds and runs fine, the repos are untouched before and after generating the vs solution.
> xmake repo -l
local repositories:
    local-repo xmake-packages
    iconeus-repo https://github.com/Iconeus/xmake-repo.git

global repositories:
    build-artifacts https://gitlab.com/xmake-mirror/build-artifacts.git main
    xmake-repo https://gitlab.com/tboox/xmake-repo.git master
    builtin-repo C:\Users\Charles\xmake\repository

5 repositories were found!

> xmake
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.31.31104
updating repositories .. ok
checking for Qt SDK directory ... C:/Users/Charles/Documents/work/qt6-build-msvc/qtbase
checking for Qt SDK version ... 6.3.0
...
[100%]: build ok!

> xmake repo -l
local repositories:
    local-repo xmake-packages
    iconeus-repo https://github.com/Iconeus/xmake-repo.git

global repositories:
    build-artifacts https://gitlab.com/xmake-mirror/build-artifacts.git main
    xmake-repo https://gitlab.com/tboox/xmake-repo.git master
    builtin-repo C:\Users\Charles\xmake\repository

5 repositories were found!

> xmake project -yk vsxmake
...
create ok!

> xmake repo -l
local repositories:
    local-repo xmake-packages
    iconeus-repo https://github.com/Iconeus/xmake-repo.git

global repositories:
    build-artifacts https://gitlab.com/xmake-mirror/build-artifacts.git main
    xmake-repo https://gitlab.com/tboox/xmake-repo.git master
    builtin-repo C:\Users\Charles\xmake\repository

5 repositories were found!

I reload the solution and build it, I get an error
error : Designtime build failed for project 'C:\Users\Charles\Documents\work\IcoStudio\vsxmake2022\IcoStudioInterface\IcoStudioInterface.vcxproj' configuration 'debug|x64'. IntelliSense might be unavailable.
  Set environment variable TRACEDESIGNTIME = true and restart Visual Studio to investigate.

I have to cancel the compilation and build again. It launches, but a configuration is missing.
xmake config -y  -P . -p windows -m debug -a x64 -o "build"
should be
xmake config -y  -P . -p windows -m debug -a x64 -o "build" "--matlabroot=D:\\Program Files\\MATLAB\\MATLAB Runtime\\v911"
Build started...
1>------ Build started: Project: IcoStudioInterface, Configuration: debug x64 ------
1>$xmake config -y  -P . -p windows -m debug -a x64 -o "build"
1>checking for Microsoft Visual Studio (x64) version ... 2022
1>checking for Microsoft C/C++ Compiler (x64) version ... 19.31.31104
1>checking for Qt SDK directory ... C:/Users/Charles/Documents/work/qt6-build-msvc/qtbase
1>checking for Qt SDK version ... 6.3.0
1>generating src\IcoStudioInterface\Config.hpp.in ... ok
1>$xmake build -y  -P . -w "IcoStudioInterface"
1>[ 85%]: compiling.debug build\.gens\IcoStudioSoftware\windows\unity_build\unity_1.cpp
1>[ 87%]: archiving.debug IcoStudioSoftware.lib
1>[ 89%]: compiling.debug build\.gens\IcoStudioInterface\windows\unity_build\unity_1.cpp
1>[ 90%]: compiling.debug build\.gens\IcoStudioRendering\windows\unity_build\unity_1.cpp
1>[ 95%]: archiving.debug IcoStudioRendering.lib
1>[ 97%]: linking.debug IcoStudio.exe
1>[100%]: build ok!
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

It builds fine but the runtime can't find the dlls provided with the matlabroot option (although it also provides include dirs, link dir and links?).
I reconfigure and rebuild the project.
> xmake config -y --matlabroot="D:\Program Files\MATLAB\MATLAB Runtime\v911"

> xmake project -yk vsxmake
...
create ok!

Then I reload the solution and build. The option is here, but the repos are messed up.
Build started...
1>------ Build started: Project: IcoStudioInterface, Configuration: debug x64 ------
1>$xmake config -y  -P . -p windows -m debug -a x64 -o "build" "--matlabroot=D:\\Program Files\\MATLAB\\MATLAB Runtime\\v911"
1>checking for Microsoft Visual Studio (x64) version ... 2022
1>checking for Microsoft C/C++ Compiler (x64) version ... 19.31.31104
1>  => install libiconeus 0.1.3 .. failed
1>
1>EXEC : error : Cloning into 'C:\Users\Charles\Documents\work\IcoStudio\.xmake\windows\x64\repositories\iconeus-repo'...
1>fatal: Remote branch master not found in upstream origin
1>if you want to get more verbose errors, please see:
1>  -> C:\Users\Charles\AppData\Local\.xmake\cache\packages\2207\l\libiconeus\0.1.3\installdir.failed\logs\install.txt
1>EXEC : error : install failed!
1>C:\Users\Charles\xmake\plugins\project\vsxmake\vsproj\Xmake.targets(97,5): error MSB3073: The command "..." exited with code -1.
1>Done building project "IcoStudioInterface.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

> xmake repo -l
local repositories:
    local-repo C:\Users\Charles\Documents\work\IcoStudio\xmake-packages
    iconeus-repo C:\Users\Charles\Documents\work\IcoStudio\.xmake\windows\x64\repositories\iconeus-repo
    local-repo C:\Users\Charles\Documents\work\IcoStudio\xmake-packages
    iconeus-repo C:\Users\Charles\Documents\work\IcoStudio\.xmake\windows\x64\repositories\iconeus-repo

global repositories:
    build-artifacts https://gitlab.com/xmake-mirror/build-artifacts.git main
    xmake-repo https://gitlab.com/tboox/xmake-repo.git master
    builtin-repo C:\Users\Charles\xmake\repository

7 repositories were found!

I remove the repos, it compiles and runs well, we're back to the initial state.
> xmake repo -r local-repo
remove local repository(local-repo): ok!
> xmake repo -r iconeus-repo
remove local repository(iconeus-repo): ok!
> xmake repo -l
local repositories:
    local-repo xmake-packages
    iconeus-repo https://github.com/Iconeus/xmake-repo.git

global repositories:
    build-artifacts https://gitlab.com/xmake-mirror/build-artifacts.git main
    xmake-repo https://gitlab.com/tboox/xmake-repo.git master
    builtin-repo C:\Users\Charles\xmake\repository

5 repositories were found!

> xmake
...
[100%]: build ok!

> xmake r
>

Now I reconfigure with forward-slashes in the option
> xmake config -y --matlabroot="D:/Program Files/MATLAB/MATLAB Runtime/v911"

> xmake project -yk vsxmake
...
create ok!

And everything works fine both from the command line and from vsxmake without any error.

xmake config -y --matlabroot="D:\Program Files\MATLAB\MATLAB Runtime\v911"

you should use \\ instead of \

No, the behavior should be consistent between xmake CLI and vsxmake

xmake config -y --matlabroot="D:\Program Files\MATLAB\MATLAB Runtime\v9

The cli terminal shell will escapes \, so you have to use \\ instead of \, which has nothing to do with xmake.
xmake already gets the wrong parameter value from the terminal, before it has done anything.
xmake has no chance to change anything, unless the user corrects the input parameters.

If xmake gets a wrong parameter value, why does it work in CLI, then?

It builds fine but the runtime can't find the dlls provided with the matlabroot option (although it also provides include dirs, link dir and links?).
I reconfigure and rebuild the project.
xmake config -y --matlabroot="D:\Program Files\MATLAB\MATLAB Runtime\v911"

Didn't you set it in cli?

Yes, I set it for the command line in powershell

Yes, I set it from the command line in powershell

so, you should use
xmake config -y --matlabroot="D:\\Program Files\\MATLAB\\MATLAB Runtime\\v911"

powetshell will escape \\, then xmake will get a valid \ character.
if you use \MATLAB, powershell will escape \M amd xmake will get a invalid character or missing it.
This is also the case in the cmd.

But it doesn't, tho

It compiles and runs fine

you can edit Xmake.Custom.props file and enable
    <!-- Set -v for all xmake call -->
     <XmakeVerbose Condition="'$(XmakeVerbose)' == ''">true</XmakeVerbose>

    <!-- Set -D for all xmake call -->
    <XmakeDiagnosis Condition="'$(XmakeDiagnosis)' == ''">true</XmakeDiagnosis>

then give me full logs for installing packages.

1>$xmake config -y  -P . -p windows -m debug -a x64 -o "build" "--matlabroot=D:\Program Files\MATLAB\MATLAB Runtime\v911"
1>checking for Microsoft Visual Studio (x64) version ... 2022
1>checking for Microsoft C/C++ Compiler (x64) version ... 19.31.31104
1>  => install libiconeus 0.1.3 .. failed
1>


Ubuntu Mate 16.10 Docker-compose up issue

Hi mate,
I'm trying to get this one up and running. Got installed docker and I'm trying to spin up the instance. This is what I'm getting:
"
d@ubuntu:~/Desktop/fuzzapi$ sudo docker-compose up
Starting fuzzapi_redis_1
Building web
Step 1 : FROM ruby:2.2
---> b068a763181d
Step 2 : RUN apt-get update && apt-get install -y   build-essential   nodejs
---> Running in c0dd0b2723fc
Err http://deb.debian.org jessie InRelease
Err http://security.debian.org jessie/updates InRelease
Err http://deb.debian.org jessie-updates InRelease
Err http://deb.debian.org jessie Release.gpg
Could not resolve 'deb.debian.org'
Err http://security.debian.org jessie/updates Release.gpg
Could not resolve 'security.debian.org'
Err http://deb.debian.org jessie-updates Release.gpg
Could not resolve 'deb.debian.org'
Reading package lists...
W: Failed to fetch http://deb.debian.org/debian/dists/jessie/InRelease
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/InRelease
W: Failed to fetch http://security.debian.org/dists/jessie/updates/InRelease
W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg  Could not resolve 'deb.debian.org'
W: Failed to fetch http://security.debian.org/dists/jessie/updates/Release.gpg  Could not resolve 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/Release.gpg  Could not resolve 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package build-essential
E: Unable to locate package nodejs
ERROR: Service 'web' failed to build: The command '/bin/sh -c apt-get update && apt-get install -y   build-essential   nodejs' returned a non-zero code: 100
"
Any ideas why is going for jessie updates?
Cheers.

@Os3r3um I have updated ruby to 2.3.0 and fixed docker issues. #44
Thanks for trying tool. Let me know if you can setup easily with new code.


checkov: 2.4.2 -> 2.4.5

Diff: https://github.com/bridgecrewio/checkov/compare/refs/tags/2.4.2...2.4.5
Changelog: https://github.com/bridgecrewio/checkov/releases/tag/2.4.5
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 250568 run on x86_64-linux 1

2 packages built:

checkov
checkov.dist