writing-a-plugin "Call the callback function only when the current file (stream/buffer) is completely consumed"

What were you expecting to happen?
Following the verbal guidance here  -

"Call the callback function only when the current file (stream/buffer) is completely consumed"

then in case of  file.IsStream() I expected to need to wait for the 'end' event from the inner transform before calling the callback().
What actually happened?
As shown in the code example on that man page, it only works if callback() is called immediately.
Please post a sample of your gulpfile (preferably reduced to just the bit that's not working)
    innerTransform.on('end', () => {
      cb(null, file) // gulp writes an empty file doing it this way
    })

Actually implementing the above code (instead of calling the callback immediately), resulted in an empty file being written, although the inner transform had dutifully pushed all the data.
What version of gulp are you using?
gulp 4
What versions of npm and node are you using?
npm 6.9.0
gulp 4.0.2

I am unable to convince myself that

when the current file (stream/buffer) is completely consumed

could mean anything other than the end event has been emitted.  Furthermore, the wording seems perfect from a design point of view, and fits with all the usage of callbacks that I know.
How to reconcile the words and the actual design?

Can you post the full plugin code?

@craigphicks I agree with @contra - there might be some weirdness here that we can only determine by seeing all the code.

Hi @contra @phated -
import through from 'through2'
import PluginError from 'plugin-error'

function makePluginError(PLUGIN_NAME, err) {
  if (err instanceof PluginError) {
    return err
  } else if (err instanceof Error) {
    let pe = new PluginError(PLUGIN_NAME, err.message)
    pe.stack = err.stack
    return pe
  } else {
    return new PluginError(PLUGIN_NAME, err.toString())
  }
}

// The purpose of this "makeOverrides" is only to convert InnerTransform errors
// to PluginError.  Necessary because the outer on-'error' handler's errors are ignored
// in favor of another listeners errors
function makeOverrides(PLUGIN_NAME, InnerTransform) {
  return {
    transform(chunk, enc, cb) {
      InnerTransform.prototype._transform.call(this, chunk, enc, (err, data) => {
        if (err)
          cb(makePluginError(PLUGIN_NAME, err), null)
        else
          cb(null, data)
      })
    },
    flush(cb) {
      InnerTransform.prototype._flush.call(this, (err, data) => {
        if (err)
          cb(makePluginError(PLUGIN_NAME, err), null)
        else
          cb(null, data)
      })
    },
  }
}

function gulpCreateOuterTransform(
  PLUGIN_NAME, InnerTransform,
  innerArgs, transformOpts) {
  //function GulpOuterTransform(...args) {
  // Creating a stream through which each file will pass
  return through.obj(function (file, enc, cb) {
    if (file.isNull()) {
      // return empty file
      return cb(null, file)
    }
    let outChunks = []
    let rppt = new InnerTransform(
      ...innerArgs,
      Object.assign((transformOpts || {}), makeOverrides(PLUGIN_NAME, InnerTransform))
    )

    rppt.on('end', () => {
      /// https://stackoverflow.com/a/52303073
      let date = new Date()
      file.stat.atime = date
      file.stat.mtime = date
      ///
      if (file.isBuffer())
        file.contents = Buffer.concat(outChunks)
      // putting the callback here will result in Gulp writing an empty file
      //cb(null, file)
    })
    rppt.on('error', (err) => {
      // in the case where file.isStream()===true,
      // there is another listener on error which does not
      // convert the err to PluginErr and passed that unconverted
      // err to the destroy function of the next stage resulting
      // in hte following PluginError not being used.
      // To workaround that,  GulpWrappedInnerTransform
      // transforms the err after every call to _transform and _flush.
      this.emit('error',
        makePluginError(PLUGIN_NAME, err)
      )
      // cb was called earlier, so can't call it here
    })

    if (file.isBuffer()) {
      rppt.on('data', (chunk) => {
        if (!(chunk instanceof Buffer))
          chunk = Buffer.from(chunk)
        outChunks.push(chunk)
      })
      rppt.end(file.contents, 'utf8') // 'end()' is equiv write() + signal end
    }
    if (file.isStream()) {
      file.contents = file.contents
        .pipe(rppt)
      rppt.resume()
    }
    cb(null, file) // return early - seems anti intuitive but written file is empty otherwise
  })
}
export default gulpCreateOuterTransform

Main Issue:
Making the callback cb(null, file) immediately seems anti intuitive.  I expected to call it from inside end and error handlers.  But when I tried that the written file was empty.  According to man page example code, calling the callback immediately is correct (see below).  However, according the man page verbal explanation "Call the callback function only when the current file (stream/buffer) is completely consumed" I feel my expected method (calling from end and error) should be correct.
Secondary issue
As recommended is some gulp instructionals, the code is using emit to signal the error, properly converted to a gulp plugin error.  This has to done because by calling the callback early, it cannot be used to return an error later. However, at the node-stream-transform level, another listener is registered for error, and even though it is second in the listener list, it is first to pass the error (an unconverted error) up the outer gulp transform level.  The workaround is the overrides for transform and flush you see near the top, used to convert the error which will be passed up by that second listener.
Caveat
The code as is works now for the case of a single file opened with src and written with dest .  I haven't tested multiple files.

I concur; I have observed the exact behavior described by @craigphicks .  I didn't realize that this conflicted with the instructions, but my notes show that I am seeing the same issue:
        .on('end', function () {

          // DON'T CALL THIS HERE. It MAY work, if the job is small enough. But it needs to
          // be called after the stream is SET UP, not when the streaming is DONE.
          // Calling the callback here instead of below may result in data hanging in the
          // stream--not sure of the technical term, but dest() creates no file, or the file
          // is blank

          // cb(returnErr, file);

        })

Per @craigphicks caveat:

The code as is works now for the case of a single file opened with src and written with dest . I haven't tested multiple files.

I too can write one file successfully, but when I write two files only one writes correctly in some circumstances. Behavior noted below is consistent given the test data I'm using, but it may well be dependent upon timing and other factors, such as the size of the data being moved, etc.
...
    else if (file.isStream()) {
      let newStream = through2.obj(function (this: any, file: Vinyl, encoding: string, cb: Function) {
        cb(null, file)
      })

      let newFile = new Vinyl({path: newFileName, contents:newStream})

      file.contents
        .pipe(request(configObj))
        .pipe(newStream)

      // In this order, both files are written correctly by dest();
      // this.push(file)
      // this.push(newFile)

      // in THIS order newFile is still written correctly, and file
      // is written but is blank. If either line is commented
      // out, the other line writes correctly
      this.push(newFile)
      this.push(file)

      // after our stream is set up (not necesarily finished) we call the callback
      log.debug('calling callback')
      cb(returnErr);
    }
...

Full source file is here, and working project is here

@donpedro - Your project is very interesting.  I have a question about
let newStream = through2.obj(function (this: any, file: Vinyl, encoding: string, cb: Function) {
        cb(null, file)
      })

When does this function (pass as argument) get called and what entity receives the data?

@craigphicks that's just an empty transformFunction; I could have just left the function out completely, as I'm not actually doing any transformation; I'm really just using newStream as a passthrough so I can use it for newFile. More here: https://www.npmjs.com/package/through2#transformfunction
Thanks for the good word! Be aware that gulp-api-adapter is very much under construction; it's being developed to work with our gulp-etl project, but hopefully it'll have some utility in the wider gulp world.
Hopefully it'll be usable and stable later this week...

I might have a smaller example for when the vinyl stream becomes empty, even though I'm not that experienced with streams.
const {dest} = require('gulp');
const source = require('vinyl-source-stream');
const size   = require('gulp-size');

function sizeOfStream(cb) {
  const stream = source('bogusname');
  stream.write('a');
  stream.end();
  stream.pipe(size())
    .pipe(dest('.'));
  cb();
}

exports.default = sizeOfStream;

The output file of the above task is empty, expected was a content of 'a'.
The solution mentioned above, calling the callback immediately (in gulp-size), without waiting for the subtask to finish, works. However, interestingly enough, it breaks the logic in the flush function (gulp-size uses through2).

@jonatanlinden You're ending the stream before you piped it anywhere - you probably want to do this:
import { pipeline } from 'stream'

// later on in your code
const stream = source('bogusname');

pipeline(
  stream,
  size(),
  dest('.'),
  cb
)
 stream.write('a');
 stream.end();

Have you actually tried your code, and gotten another result? When I run it, I get exactly the same behaviour as with my example: with size(), bogusname is empty, without it, it is not.
(I believe stream.end() indicate that no more data will be written to the stream, but it does not really affect the readable part of the stream. Maybe?)
The point was that gulp-size is also written with the understanding that the callback function should be called after the vinyl content's stream has been processed. To me, that seems what this issue is about.
I guess it could be possible to construct an even smaller example exposing the behaviour, using gulp-size as a starting point.

I was having a very similar issue with a plugin that invokes child_process.spawn.  The gulp task is "finished" in a few ms, even though the child process hasn't started yet and nobody had invoked the callback.
import {spawn} from 'node:child_process';
import process from 'node:process';

import gulp from 'gulp';
import PluginError from 'plugin-error';

function generateSources() {
  const nodeExecutable = process.argv[0];

  return through.obj((file, encoding, callback) => {
    if (file.isNull()) {
      callback(null, file);
      return;
    }

    // stdin and stdout are pipes, and stderr is inherited from this process.
    const child = spawn(nodeExecutable, ['dist/tools/code-gen.js'], {
      stdio: ['pipe', 'pipe', 'inherit'],
    });

    if (file.isStream()) {
      console.log('Is stream.');
      // file.contents is a ReadableStream
      file.contents.pipe(child.stdin);
    } else if (file.isBuffer()) {
      console.log('Is buffer.');
      // file.contents is a Buffer
      child.stdin.write(file.contents);
      child.stdin.end();
    }

    // Now file.contents is a ReadableStream from the child process's stdout.
    file.contents = child.stdout;

    // The filename should end in .ts.
    file.extname = '.ts';

    // If the child fails, this pipe fails.
    child.on('error', (error) => {
      console.log('On error.', error);
      callback(error, null);
    });

    // Wait for the child to complete.
    // Note that 'exit' fires before the pipes close, so we use 'close' to make
    // sure we have all the data.
    child.on('close', (code) => {
      console.log('On close.', code);
      if (code == 0) {
        // Exit code zero, this pipe succeeds.
        callback(null, file);
      } else {
        // Exit code non-zero, this pipe fails.
        const error = new PluginError(
            'GenerateSources', `Failed with exit code ${code}`);
        callback(error, null);
      }
    });
  });
}

All my logs would occur after the task "finished" "successfully", even if I deliberately broke the command to trigger ENOENT.
I finally found the cause.  It was my task:
gulp.task('generate-sources', gulp.series('build-tools', async () => {
  return gulp.src('src/gen/*.yaml')
    .pipe(generateSources())
    .pipe(gulp.dest('generated/gen/'));
}));

The problem was async in my task definition.  The fixed version works correctly:
gulp.task('generate-sources', gulp.series('build-tools', () => {
  return gulp.src('src/gen/*.yaml')
    .pipe(generateSources())
    .pipe(gulp.dest('generated/gen/'));
}));


[stdlib][performance] skip copying old values during removeAll(keepCapacity: true)

(This is a re-submission of #477 with clearer patch and description.)
For .removeAll(keepCapacity: true) on a non-uniquely referenced Dictionary or Set, the existing code implements COW by calling ensureUniqueNativeStorage(), which actually copies the content of its storage. Only then the elements get deleted one by one.
A FIXME(performance) comment stated this wastefulness (benchmark result posted in #477 agrees). A better solution is directly replace the native storage with equal capacity.
A note on Set: As @dabrahams pointed out, aSet.removeAll() is the equivalent of aSet.intersectInPlace([]). Ideally, intersectInPlace(), being the more general case, should be the target for optimization. In successfully doing so, the equivalence in math can be directly realized in code.
In that light, this patch can be treated as solely an optimization for Dictionary. The speedup for Set is a side-effect (courtesy of GYP and architect of this file).
Tests in both swift/validation-test/stdlib/Set.swift and ./lit swift/validation-test/stdlib/Dictionary.swift are passing.

@dduan Can you please write a simple benchmark that verifies that things go faster now?

@nadavrot Here
unoptimized:
23.2448832392693

optimized:
0.0532280802726746

@dduan Excellent!!

Your thoughts, @gribozavr ?

Only a 438x improvement?  I don't know, perhaps we should pass on this. :-)
+1 from me, but @gribozavr or @dabrahams need to approve it.

@dduan Looks great, thank you!


Python Batch API fails to create pool without indication of error whent trying to use offer centos 7-7

Problem Description
When using the Python Batch API with following Offer:
publisher="microsoft-azure-batch",
offer="centos-container",
sku="7-7",
version="latest"

virtual_machine_configuration=batchmodels.VirtualMachineConfiguration(
            image_reference=batchmodels.ImageReference(
                #virtual_machine_image_id = config._POOL_VM_IMAGE
                **publisher="microsoft-azure-batch",
                offer="centos-container",
                sku="7-7",
                version="latest"**
            ),
            container_configuration=batch.models.ContainerConfiguration(
                container_image_names=[config._CONT_IMAGE_NAME],
                container_registries=[batchmodels.ContainerRegistry(
                    user_name=config._ACR_ACCOUNT_NAME,
                    password=config._ACR_ACCOUNT_KEY,
                    registry_server=config._ACR_REGISTRY_SERVER)]
            ),
            node_agent_sku_id="batch.node.centos 7"
            ),
        vm_size=config._POOL_VM_SIZE,
        target_dedicated_nodes=4,#(0 if config._POOL_AUTO_SCALE else config._POOL_NODE_COUNT),
        target_low_priority_nodes=0,
        task_slots_per_node=config._POOL_NODE_SLOTS,
        task_scheduling_policy = batchmodels.TaskSchedulingPolicy(node_fill_type = 'pack'),
        start_task=batchmodels.StartTask(
            command_line="/bin/bash -c \"echo test\"",
            wait_for_success=True,
            user_identity=batchmodels.UserIdentity(
                auto_user=batchmodels.AutoUserSpecification(
                    scope=batchmodels.AutoUserScope.pool,
                    elevation_level=batchmodels.ElevationLevel.admin))),
        network_configuration=batchmodels.NetworkConfiguration(
            subnet_id=config._POOL_VNET,
            public_ip_address_configuration=batchmodels.PublicIPAddressConfiguration(
                provision='usermanaged',
                ip_address_ids=[ free_pips[i], free_pips[i+1] ]
            ))

Pool is NOT created but there is no indication of error at all.
Code that calls this has been unchanged and was working for months.
Upong investigating we now learn that : list_supported_images()
only returns following offer: centos-container | microsoft-azure-batch | _ 7-8_ | latest | batch.node.centos 7
Steps to Reproduce

Use Python Batch API azure_batch-12.0.0-py2.py3-none-any.whl
Try and create pool
Silent failure with Batch Explorer only reporting:

-list_supported_images() no longer mentions 7-7
Expected Results
I would expect error responses which indicate a given offer is no longer available
Actual Results
-Unable to create pool
-No intelligible error messages
Additional Comments
If this is not a bug then it would be a feature enhancement to improve the informational nature of error messages.

@KrisJanssen please send me an email at<EMAIL_ADDRESS>- I looked at your pool and the issue is likely not image related and I do not want to post details on GitHub

As a side note: centos 7.7 is out of date, we still allow you to start it but as list_supported_images() says, 7.8 is the current state of the arts. We will be releasing 7.9 shortly.

As a side note: centos 7.7 is out of date, we still allow you to start it but as list_supported_images() says, 7.8 is the current state of the art. We will be releasing 7.9 shortly.

Thanks for your rsponses! I have provided feedback via mail. It is indeed good these images continue to get updates as we indeed appreciate images might go out of date/require updates. However, is there a good resource to get advance notice of this? Thanks!

Looks like it was a configuration error, please follow up with me via e-mail if you need further assistance.


Finalize

Initial attempt to resolve #333

This fmt issue may have something to do with the Rust version, aka

is it 1.48.0?
is it stable?

Because some other versions, and some nightly, have different cargo fmt preferences.

let me cc @Pratyush and @ValarDragon here for some feedback on the API.

It seems that all the comments have been resolved, with this one needing more discussion:
whether or not to keep this function
fn reduce_constraint_weight(&mut self) {

Since this function is now fn rather than pub fn, I think for now it is okay to remove this function. The function is only, previously used by Marlin (who needs to change accordingly).
Nick, what do you think?

It seems that all the comments have been resolved, with this one needing more discussion:
whether or not to keep this function
fn reduce_constraint_weight(&mut self) {

Since this function is now fn rather than pub fn, I think for now it is okay to remove this function. The function is only, previously used by Marlin (who needs to change accordingly).
Nick, what do you think?

Yes, that makes sense!

Yes, that makes sense!

Let me cc @Pratyush and @ValarDragon for a final pass

Let me cc @Pratyush and @ValarDragon for a final pass

I left a comment about None

I left a comment about None

Based on the discussion above, I think we will keep None for now and redirect it to self.inline_all_lcs(). This is the last change needed/ In the future, different treatments may occur, such as reducing the amount of symbolic LC.

Based on the discussion above, I think we will keep None for now and redirect it to self.inline_all_lcs(). This is the last change needed/ In the future, different treatments may occur, such as reducing the amount of symbolic LC.

Checks pass :) Pratyush and Dev, if it looks good, feel free to merge or do some final edits.

Checks pass :) Pratyush and Dev, if it looks good, feel free to merge or do some final edits.

One more: we need to export OptimizationGoal here:
https://github.com/npwardberkeley/snark/blob/finalize/relations/src/r1cs/mod.rs#L21
Otherwise, Marlin could not find it.

One more: we need to export OptimizationGoal here:
https://github.com/npwardberkeley/snark/blob/finalize/relations/src/r1cs/mod.rs#L21
Otherwise, Marlin could not find it.

add one to enforce that set_optimization_goal must check that no constraint/variable has been written.
        num_instance_variables: 1,
        num_witness_variables: 0,
        num_constraints: 0,
        num_linear_combinations: 0,

(sorry to bother Nick---I could add them if you add me as a PR repo's writer)

add one to enforce that set_optimization_goal must check that no constraint/variable has been written.
        num_instance_variables: 1,
        num_witness_variables: 0,
        num_constraints: 0,
        num_linear_combinations: 0,

(sorry to bother Nick---I could add them if you add me as a PR repo's writer)


error<EMAIL_ADDRESS>proxy: COCO_PROXY='true' nodemon

npm run proxy
then get the error
error<EMAIL_ADDRESS>proxy: `COCO_PROXY='true' nodemon`

Hi @aabababb - that's unfortunate. Were you able to fix this?
We've just updated the onboarding guide to include a docker guide which is much easier to get started with, and includes all the dependencies you need. That may be helpful!

How do I fix this? COCO_PROXY is not a command no matter what i do.


Default Gateway Suggestion

Using <IP_ADDRESS> as the default gateway would make it easier for those using VMware Workstation. VMware workstation defaults to using .1 as the host-to-vm virtual adapter so the Gateway needs to be modified to a number between .2-.254.

My gateways are always at 254, primarily because that is what I always encountered in the past. Or perhaps there needs to be some sort of preference setting for things like this as well as the domain name.

IP addressing is up to the Author and Student - I expect people to have different IP address needs.  Do you think its too hard for folks to change the PSD1?  The Pluralsight configs will be labeled for each course and can have custom IP addresses and domains -- whatever the Author wants.  Does that help?

If @themichaelbender @jdhitsolutions @majst32  agree that authors can customize the addresses themselves (as by design) in the psd1, I would like to close this issue.

As long as documentation for end users includes how to update this, I am fine with closing this. Also, setup video/module should detail this process.

@themichaelbender @jdhitsolutions @majst32 - well, let me make sure I'm clear -- The POC (Proof of Concept) configs are examples for the author.  The author will configure whatever they want for the course, and add a folder with the course ID with their own configs, IP addresses, computer names and such.  Students should not ever need to open these files -- that is the point --- unless they are learning DSC.  I will provide the students with a video on how to "Run" the Autolab and select the correct course, but I'm not going into the configs.  Does that make sense?  This is really an author tool.  And while I can help an author troubleshoot their own configs, its really up to the author to write their own and create the environment they want for the student.


Reconstruction Target

Hi,
Thanks for your awesome work and opensource.
I have a question about the reconstruction target. Compared with MAE which reconstructs the whole image as output, PointMAE seems to only reconstruct points that are not masked. (with shape mnk3). However, it seems that PointMAE reconstructs the whole point from Figure 2.
I am confused about the reconstruction target. Please tell me if I missed something or had a wrong understanding.

Hi,
First, MAE only reconstructs masked patches, referring to their paper in Sec. 3 Reconstruction target. Our Point-MAE also reconstructs masked point patches. m is the masking ratio. Therefore, mnxkx3 represents masked point patches. We have clarified the reconstruction target in Section 3.3.
In Figure 2, we visualize the combination point cloud of predicted point patches and visible point patches, in order to compare with the input point cloud.
Sorry for the confusion.

MAE also reconstructs masked patches since its loss is computed only on the masked parts, then combines predicted and visible patches to produce final images.


add pipenv shell --run

when im creating a subshell i want to give it an initial commands or a list with initial commands
for example i want to run ptpython in a shell script after launching subshell like this:
#!/usr/bin/zsh
pipenv shell # this runs, but
echo "test" # this will be executed after the subshell its killed
ptpython # same to this

the feature would look like this
pipenv shell --init-cmd="ptpython"

4 running a single command
and
pipenv shell --init-cmd="echo'test'; ptpython; echo 'i l0ve this fe@ture'"

the commands will be splitted by semicolon(;)
and also a very good idea, give it an initial script to run after launching subshell
pipenv shell --init-script=init.zsh

assuming init.zsh is at cwd
or full path
pipenv shell --init-script=/home/user/path/folder/init.zsh

nevermind.
this feature is already present
you run a custom command like this
pipenv shell ptpython

or you can run a script like this
pipenv shell ./tmux-scripts/init-ipy.zsh


Cleanup user guide sections a-c

Account management, container preloading, custom CI, and custom SOTA client sections were all cleaned up in line with the style guide. Container preloading was also updated to remove references of named branches (devel), and removed a number of screenshots of the UI which are covered in the getting started and tutorial sections.
QA Steps: viewed rendered html in browser. Ran vale linter and linkcheck.
This commit applies to FFTK-988
Readiness

[x] Merge (pending reviews)
[ ] Merge after date or event
[ ] Draft

Overview
Just some basic cleanup and minor updating.
Checklist

[x] Run spelling and grammar check, preferably with linter.
[x] Avoid changing any header associated with a link/reference.
[ ] Step through instructions (or ask someone to do so).
[x] Review for wordiness
[x] Match tone and style of page/section.
[x] Run make linkcheck.
[x] View HTML in a browser to check rendering.
[x] Use semantic newlines.
[x] follow best practices for commits.

[x] Descriptive title written in the imperative.
[x] Include brief overview of QA steps taken.
[x] Mention any related issues numbers.
[x] End message with sign off/DCO line (-s, --signoff).
[x] Sign commit with your gpg key (-S, --gpg-sign).
[x] Squash commits if needed.

[x] Request PR review by a technical writer and at least one peer.

Comments
If any major changes need to be made to the content to reflect recent changes, the can be handled as a separate commit; the focus here is largely on style.

I will fix that build error shortly!

Docs for 1a09e8229ce76275168f459957db70b8a547773c are browsable at: https://ci.foundries.io/projects/fio-docs/builds/2317/docs/artifacts/html/index.html

Docs for 8d4cf8f31f8648ac01276267a80cc402b2032a80 are browsable at: https://ci.foundries.io/projects/fio-docs/builds/2318/docs/artifacts/html/index.html


chore(main): Enable release-trigger bot for triggering release job

Changes made in accordance with this doc.
A common build file, shared by all the Go split repos, is created for the release job.

@diptanshumittal thanks for bringing this to our attention. So ECP actually has a kokoro pipeline that uses copybara to mirror this github repo to google3, builds/signs/notarizes the artifacts, and uploads them to GCS buckets. We then manually upload the final artifacts to the Github repo's versioned release page. I'm wondering if the EO can be satisfied without introducing additional scripting complexities to the github repo itself. WDYT?

@diptanshumittal thanks for bringing this to our attention. So ECP actually has a kokoro pipeline that uses copybara to mirror this github repo to google3, builds/signs/notarizes the artifacts, and uploads them to GCS buckets. We then manually upload the final artifacts to the Github repo's versioned release page. I'm wondering if the EO can be satisfied without introducing additional scripting complexities to the github repo itself. WDYT?

Thanks for the information @andyrzhao. Its great to know we already have a kokoro pipeline for mirroring the repo. Although, it won't be able to satisfy the EO completely, we should be able to make it work with slight modification.


Impossible to use schemas

Generated quotation marks around table names (CREATE TABLE 'mytable') make it impossible to use schemas, as
CREATE TABLE 'myschema.mytable';

will actually create a table with a dot in public schema.

I think it is good behavior. Schema can be passed as an argument or like:
pgm.createSchema('myschema', function (pgm) {
  pgm.createTable('mytable');
});

I haven't really used schemas at all. Can I just drop the quotes and will everything be ok?

Can I just drop the quotes and will everything be ok?

No this will not work as the pgm.createTable expects a string.
One solution would be to add an extra argument to createTable (and other pgm functions) or include the schema in the DATABASE_URL.

oh I meant could I just drop the quotes in the generated sql?
In regards to setting the schema in the URL or config somehow, do you think it is safe to assume that everything a user would want to do would be within a single schema?

oh I meant could I just drop the quotes in the generated sql?

Yes that would do it as @punund points out at the top of the thread. I tested it too by modifying line 15 in lib/operations/tables.js to this:
var sql = utils.t("CREATE TABLE {table_name} (\n{columns}\n){inherits};", {

In regards to setting the schema in the URL or config somehow, do you think it is safe to assume that everything a user would want to do would be within a single schema?

The reason for suggesting setting the schema in the DATABASE_URL would be to follow how db-migrate does it. However assuming someone would want all their migrations in a single schema is a real corner case. Best options would just be to have it as an argument to a pgm create function.

I would not drop quotes as it Postgres is case sensitive and "pg" and "PG" are two different identifiers.
Using wrapper function as @sheerun suggest is also not good option, because you may want to do cross schema references in some queries.
So I implemented in a way, that you can now pass object as a name with keys schema and name and it will produce correct queries. (https://github.com/theoephraim/node-pg-migrate/blob/fb16636360430a8897e058947f2f857bdf85ed5e/README.md#using-schemas)
Is it convenient for you?

Solved in PR #58.


Address in use errors

---- net::transport::e2e_tests::fails_if_not_subscribed stdout ----
thread 'net::transport::e2e_tests::fails_if_not_subscribed' panicked at 'Failed to serve: Os { code: 98, kind: AddrInUse, message: "Address already in use" }', src/net/server/mod.rs:77:26
thread 'net::transport::e2e_tests::fails_if_not_subscribed' panicked at 'Failed to bind server to a port', src/net/server/mod.rs:95:14

I believe we haven't seen this in a long time, so I'm going to close this issue.


Appstudio update test-component-pac-ggki

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI/test-component-pac-ggki-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
5 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
30 seconds
sast-snyk-check

✅ Succeeded
57 seconds
build-container

✅ Succeeded
24 seconds
inspect-image

✅ Succeeded
28 seconds
deprecated-base-image-check

✅ Succeeded
49 seconds
clamav-scan

✅ Succeeded
4 minutes
clair-scan

✅ Succeeded
16 seconds
sbom-json-check

✅ Succeeded
6 seconds
label-check

✅ Succeeded
6 seconds
show-sbom

✅ Succeeded
6 seconds
show-summary

Pipelines as Code CI/test-component-pac-ggki-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
5 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
30 seconds
sast-snyk-check

✅ Succeeded
57 seconds
build-container

✅ Succeeded
24 seconds
inspect-image

✅ Succeeded
28 seconds
deprecated-base-image-check

✅ Succeeded
4 minutes
clair-scan

✅ Succeeded
49 seconds
clamav-scan

✅ Succeeded
16 seconds
sbom-json-check

✅ Succeeded
6 seconds
label-check

✅ Succeeded
6 seconds
show-sbom

✅ Succeeded
6 seconds
show-summary

Pipelines as Code CI/test-component-pac-ggki-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
6 seconds
init

✅ Succeeded
19 seconds
clone-repository

✅ Succeeded
11 seconds
sast-snyk-check

✅ Succeeded
1 minute
build-container

✅ Succeeded
13 seconds
inspect-image

✅ Succeeded
10 seconds
deprecated-base-image-check

✅ Succeeded
13 seconds
clair-scan

✅ Succeeded
45 seconds
clamav-scan

✅ Succeeded
13 seconds
sbom-json-check

✅ Succeeded
7 seconds
label-check

✅ Succeeded
7 seconds
show-sbom

✅ Succeeded
7 seconds
show-summary

Pipelines as Code CI/test-component-pac-ggki-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
6 seconds
init

✅ Succeeded
19 seconds
clone-repository

✅ Succeeded
11 seconds
sast-snyk-check

✅ Succeeded
1 minute
build-container

✅ Succeeded
13 seconds
inspect-image

✅ Succeeded
13 seconds
clair-scan

✅ Succeeded
10 seconds
deprecated-base-image-check

✅ Succeeded
45 seconds
clamav-scan

✅ Succeeded
13 seconds
sbom-json-check

✅ Succeeded
7 seconds
label-check

✅ Succeeded
7 seconds
show-summary

✅ Succeeded
7 seconds
show-sbom


[CodingStandard] Fatal error in DynamicPropertySniff

 [ERROR] Argument 2 passed to Symplify\CodingStandard\SniffTokenWrapper\ClassWrapper::createFromFileAndPosition() must
         be of the type integer, boolean given, called in
         <project path>/vendor/symplify/coding-standard/src/Sniffs/Property/DynamicPropertySniff.php on line 81

I dumped $this->file in the getClassWrapper method and it turns out that it is a file with trait. So it can't find a T_CLASS token there obviously.
By the way a T_CLASS token could actually be there somewhere if I used an anonymous class inside of that file. That might cause more problems.

Also I'm still on the 2.5 branch because I can't use SF 4.0 for now.

Thanks for the report. Could you provide minimal PHP code causing this error?
Like this:
trait SomeTrait
{
    public function someMetho()
    {
        $this->missingProperty;
    }
}

I can fix that in current master, then you can backport it to 2.5 branch.

Will try tomorrow.

This seems to be the minimal code:
<?php declare(strict_types = 1);

trait Payload
{
	protected $payload = [];

	protected function setPayload(array $payload): void
	{
		$this->payload = $payload;
	}
}

Great job!
Could you send that as failing test to the master here as correct4.php.inc file? I'll pull and fix it then much easily

I've added test case and it is working on master.
Just look to ClassWrapper and try to apply same code to 2.5:  https://github.com/Symplify/Symplify/blob/a4540e1178b5c8ad9419c4e48b105c43b97109e9/packages/TokenRunner/src/Wrapper/SnifferWrapper/ClassWrapper.php#L47


fix: pay now button enable fix in case of wallets

Type of Change

[X] Bugfix
[ ] New feature
[ ] Enhancement
[ ] Refactoring
[ ] Dependency updates
[ ] Documentation
[ ] CI/CD

Description
SDK pay now button was getting enabled if any wallet is rendered even if we do not want it enabled before validation, or if we switch from saved payment methods to main payment method section after clicking on any wallet on saved payment methods section
How did you test it?
configuring wallets and rendering the SDK pay now button after implementing the fix and by switching between the saved payment methods and main payment methods.
Checklist

[X] I ran npm run re:build
[X] I reviewed submitted code
[ ] I added unit tests for my changes where possible

Now this PR is not required as the use case has been changed and Pay now button will always be enabled.
Closing this PR.


[Question]: how to combine milvus with my custom node ingestion pipeline

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question

nodes = meta_data2_nodes(documents)
storage_context = StorageContext.from_defaults()
storage_context.docstore.add_documents(nodes)
vector_index = VectorStoreIndex(nodes, storage_context=storage_context)
keyword_index = SimpleKeywordTableIndex(nodes, storage_context=storage_context)

meta_data2_nodes is my custom metadata extractor function. now as my data amount is quite large, i want use milvus as my vector store backend, and here is my code

documents = SimpleDirectoryReader("./data/TEST_V1/Antibody").load_data()
nodes = meta_data2_nodes(documents)
# initialize storage context (by default it's in-memory)
storage_context = StorageContext.from_defaults()
storage_context.docstore.add_documents(nodes)
# 创建向量索引（使用 Milvus）
vector_index = VectorStoreIndex.from_vector_store(vector_store=milvus_store,
storage_context=storage_context)
# 创建关键词索引（使用默认存储）
keyword_index = SimpleKeywordTableIndex(nodes, storage_context=storage_context)

my question is without passing nodes to  vector_index = VectorStoreIndex.from_vector_store(vector_store=milvus_store,                                                       storage_context=storage_context) , can i still get the correct vector index in milvus (nodes info, embedding info)

Hey @ZJL0111! 👋 I'm here to help you out with any bugs, questions, or contributions you might have. Let's tackle this together!
Yes, you can get the correct vector index in Milvus without passing nodes to VectorStoreIndex.from_vector_store. You can create a VectorStoreIndex directly from a MilvusVectorStore without needing to pass nodes. Here's an example:
from llama_index.vector_stores.milvus import MilvusVectorStore
from llama_index.core import VectorStoreIndex

vector_store = MilvusVectorStore(
    overwrite=False,
    uri="your_milvus_uri",
    token="your_milvus_token",
    collection_name="your_collection_name",
)

index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

This approach initializes the index using the existing data in Milvus, so you don't need to pass nodes explicitly [1].
To continue talking to Dosu, mention @dosu.

thanks! Dosu


Some index files failed to download. They have been ignored, or old ones used instead.

 They have been ignored, or old ones used instead.

The reason is, very likely you have old buildstep based on Ubuntu 12.04 and it's no longer supported. You need to update buildstep, check details here.

Gotcha, thanks


Release updates

fixed visualization of flows with zero length
added support to filter by conversations not passing through a common node
added support for visualization of visiting information on WA workspace chart

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Renamed Gabriella folder, relocated Jean-Paul folder to groups-bio and renamed it w/o uppercase

There was an issue before that I can't merge my branch due to failed CI check on lint:ls.
After able to find the problem with is Jean Paul folder was written with uppercase. I changed the name without uppercase.

[ ] Open the master repo on VSCode
[ ] git pull to get the recent repo version
[ ] git checkout lohygabriellato working on branch called lohygabriella
[ ] Renamed Bio folder (from me Gabriella) to lohygabriella
[ ] Relocated Jean Paul's bio to groups-bio folder and renamed his file without using uppercase due to failed npm check for lint:ls
[ ] git add .to submit the changes
[ ] Run the npm: npm run format``npm run format:check``npm run spell-check npm run lint:ls npm run lint:md
[ ] All CI check passed
[ ] git commit -m "notes why/which file I changed"
[ ] git push origin lohygabriella

Normally you should have 1 folder only for all the .md files and inside that folder, you should have only .md files and a common folder to place the images. I would use the bug label for this issue.


Support renaming existing snapshots

Hi,
We have a lot of VMs that need to have their snapshots renamed and re-creating all those snapshots seems a bit overkill ... would be nice if this was possible through the web UI!

@toby1984  This ability is currently not supported and therefore this is a wrong tracker since it  should be opened as ovirt-engine RFE at Bugzilla: https://bugzilla.redhat.com

@toby1984 I'm closing this issue since it should be opened on Bugzilla


[Bug report] plugin-active-header-links trigger route.path changes

Description
When users scrolling pages, all watches with route.path  deps shall not be triggered, however @vuepress/plugin-active-header-links break this. Every computed value and watches with route.path will be recalculated or rerun when hash changes or page scrolling to top.
Reproduction
https://stackblitz.com/edit/vuepress-ewt9uy?file=package.json,docs%2Fguide%2FREADME.md%3AL40,package-lock.json
Used Package Manager
npm
System Info
Not related

Could you explain why force: true is needed when changing hash? @meteorlxy

This should have been fixed.


Problems with ensuring proper version to be passed as parameter.

When using ensure in form:
package:
  ensure: 1.1.1

Chocolatey calls for:
C:\ProgramData\chocolatey\bin\choco.exe upgrade package -version 1.1.1 -y
which gives us:

Error: Could not update: Execution of 'C:\ProgramData\chocolatey\bin\choco.exe upgrade package -version 1.1.1 -y ' returned 3010: Plted in error (converted to warning):
Cannot bundle unregistered option '-e'.

and on every run gives notify:

Notice: /Stage[main]/Win_manage/Package[package]/ensure: ensure changed '1.1.1' to ' 1.1.1'
When using manually:

C:\ProgramData\chocolatey\bin\choco.exe upgrade package --version 1.1.1 -y
Package passes properly.
The problem is with -version instead of --version
Chocolatey in 0.10.3 version.

Are you using chocolatey/chocolatey or puppetlabs/chocolatey?
The actually error is 3010 - which means you need to pass --ignore-package-exit-codes in the install_options. Or if you use puppetlabs/chocolatey, that is already handled for you.

Thanks @ferventcoder , so another argument for switching to puppetlabs/chocolatey.

For now at least. 👍

The ultimate plan is to have this one on the edge of new features, etc, but right now it is in need of playing catch up with the work that has went into the puppetlabs/chocolatey version.

Let me know if it still has the secondary issue once you've flipped. Oh by the way, it is a drop in replacement. You won't need to change anything else.

If switching to puppetlabs/chocolatey will do - I'll do if. If not, I'll add "--ignore-package-exit-codes" as resource defaults to never, ever, ever remember of it.

@marcinbojko other handy things in the puppetlabs/chocolatey module:

chocolateysource
chocolateyconfig
chocolateyfeature

Yup. But I wrote this part earlier, for myself, using chocolatey/chocolatey. And it works. And changing a lots of hosts is a pain. And so on ;)

Fair, I would have assumed you would update your master and let it sync down the changes for you automatically.

No worries, glad we had options to move you forward without a switch over.

That part is yet to be determined. Testing now.

As usuall  @ferventcoder - what would I do without you and Chocolatey? ;)

Debug: Executing: 'C:\ProgramData\chocolatey\bin\choco.exe upgrade powershell -version 5.1.14409.1005 -y  --ignore-package-exit-codes'
Notice: /Stage[main]/Win_manage/Package[powershell]/ensure: ensure changed '5.0.10586.20161216' to '5.1.14409.1005'�����������������������������������������������������������������������������������


nix.tests.compareEval

Description of changes
Instantiate all of Nixpkgs when Nix is updated.
Succeed only when the resulting derivations are the same as Nix 2.13.
This test will also be used in upstream CI.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Without having reviewed it yet, I want to say that I very much appreciate this change!

The only goal for this PR is to test Nix; not Nixpkgs. The whole of Nixpkgs mostly only serves as test data here.

shouldn't the evaluation machinery be generalized out of the test directory and be shared with the evaluation entrypoints for all of nixpkgs

I don't know. It would be nice to have an recognizable interface into this logic, so that Nix upstream can reuse it without it being broken every now and then.
On the other hand, this stuff is rather ad hoc and not something I think people want to use, unless they're developing or maintaining Nix, or an alternate implementing like tvix, I suppose, so maybe it's not worth the effort of formalizing everything, and then having more inertia later. It's easy enough to adapt when something does change.

be shared with the evaluation entrypoints for all of nixpkgs, cc @lilyinstarlight with whom we had discussions

I'm not always up to date with the Nixpkgs Architecture Team room, but did you share this project there? You could probably get some help that way.

Just to be sure, this idea of performing sharding has first surfaced in an Adam Joseph's PR

I didn't follow that project much, and I reusing that work didn't seem like a good idea because I'd have to communicate with a banned contributor, and I would rather familiarize myself with Nix's intricacies than his implementation's intricacies, because the prior provides a more valuable experience as a Nix maintainer, but to be honest I'm not sure if I even put that much thought into the decision to only look at merged work to build on. Maybe I was just dumb. I did this work 1.5 months ago, so please forgive me for having to reverse engineer my thought process. I only continued it this week because my nix-diff PR got merged. Diagnostics are pretty important, and I have a lot on my plate, so ended up waiting for that.

think it's good to credit them somewhere in the file

I haven't used his unmerged work, and we don't normally credit the authors of the code we reuse. I guess I could mention it in a commit message, as prior art or something, but I don't even know what to say, except something "Thanks to Adam Joseph for figuring out all-attrs-eval-under-tryEval, which was a great starting point for producing a list of Nixpkgs attributes".

His (brilliant) idea was to accelerate further the evaluation of nixpkgs using sharding and GNU parallel, which was an implementation of nix-eval-jobs in pure Nix + 1 Bash utility.

GNU parallel is a pretty obvious choice for a shardable workload. Maybe it's brilliant; I haven't seen it.
Anyway, I think using separate derivations is sufficient. For what it's worth, it gives us distributed evaluation through remote builds; not just local parallelism.

I'm not always up to date with the Nixpkgs Architecture Team room, but did you share this project there? You could probably get some help that way.

And the fact that even me feels a bit offended about your response will definitely not give anyone the desire to venture in that room, it is a sad thing w.r.t. new contributors.

I was just asking and trying to be helpful. I suppose this could be read in a condescending tone. That's not at all what I was going for.
Talking to more people who care about such issues just seemed like good idea.

I don't know where did you get the information they were a banned contributor, but they are not.

Oh my, oh my, was I wrong. He took a break on his own accord, and wasn't banned at all.
I don't know how I managed to misremember this... I'll correct that right away.

I am a bit disturbed by the methodology you are hinting at

My methodology was just "solve one problem", open a PR and take it from there.
Whether that's simple or simplistic is your call, but know that I have no intent, and I do care about the big picture.
I am very sorry that I made a mistake earlier by ignoring his work, and I apologize for making you spend a whole lot of energy that should not have been necessary.
I'll take a few steps back and re-do this, or parts of this, or something similar, properly, in a new PR.
Thank you honestly very very much for your review; otherwise I would have remained oblivious to this serious mistake.


Quadrigacx

The same thing is happening with quadrigacx and its private api
    signature = self.hmac (self.encode (request), self.secret)
  File "C:\Program Files (x86)\Python36-32\lib\site-packages\ccxt\__init__.py", line 389, in hmac
    h = hmac.new (secret, request, algorithm)
  File "C:\Program Files (x86)\Python36-32\lib\hmac.py", line 144, in new
    return HMAC(key, msg, digestmod)
  File "C:\Program Files (x86)\Python36-32\lib\hmac.py", line 42, in __init__
    raise TypeError("key: expected bytes or bytearray, but got %r" % type(key).__name__)
TypeError: key: expected bytes or bytearray, but got 'str'

And just in case you didnt see it I'l post it again.
I have a problem with signature in cex.
Not sure from which side is it.Mine or yours.
Im testing for method:  get_myfee/   in cex
This is what i get as a response
{'e': 'get_myfee', 'error': 'Invalid signature', 'ok': 'error'}
Api key and secret should be correctly inputed.I have checked.Might be a problem with computing the signature?

And just in case you didnt see it I'l post it again.

I have a problem with signature in cex.
Not sure from which side is it.Mine or yours.
Im testing for method: get_myfee/ in cex
This is what i get as a response
{'e': 'get_myfee', 'error': 'Invalid signature', 'ok': 'error'}
Api key and secret should be correctly inputed.I have checked.Might be a problem with computing the signature?

I replied to your question about CEX there on your issue https://github.com/kroitor/ccxt/issues/31
I'll duplicate that here:
You probably forgot to set the uid. CEX.IO requires that. See API Keys Setup section in the manual.
To test if this is the case with uid, try the following code:
import ccxt
cex = ccxt.cex ({
    'apiKey': 'eqCv267WySlu577JnFbGK2RQzIs', # this is standard
    'secret': 'pZnbuNEm5eE4W1VRuFQvZEiFCA', # this is also standard
    'uid': 'up105393824', # for CEX.IO you should set uid as string

})
cex.fetch_balance ()

{'timestamp': '1500262307', 'username': 'up105393824', 'BTC': {'available': '0.00000000', 'orders': '0.00000000'}, 'USD': {'available': '0.00', 'orders': '0.00'}, 'EUR': {'available': '0.00', 'orders': '0.00'}, 'GBP': {'available': '0.00', 'orders': '0.00'}, 'GHS': {'available': '0.00000000', 'orders': '0.00000000'}, 'RUB': {'available': '0.00', 'orders': '0.00'}, 'ETH': {'available': '0.00000000', 'orders': '0.00000000'}}

Please, test the code above, then try your credentials and report if everything is ok or not. Thank you very much! Appreciate your help!
As for QuadrigaCX, fixing it right now, should be ok very soon.

QuadrigaCX should work now, code for testing:
import ccxt
quadrigacx = ccxt.quadrigacx ({
    'apiKey': 'jKvWkMqrOj', # this is standard
    'secret': 'f65a2e3bf3c73171ee14e389314b2f78', # this is also standard
    'uid': '395037', # you should set uid as string
})
quadrigacx.fetch_balance ()

{'btc_available': '0.00000000', 'btc_reserved': '0.00000000', 'btc_balance': '0.00000000', 'etc_available': '0.00000000', 'etc_reserved': '0.00000000', 'etc_balance': '0.00000000', 'eth_available': '0.00000000', 'eth_reserved': '0.00000000', 'eth_balance': '0.00000000', 'cad_available': '0.00', 'cad_reserved': '0.00', 'cad_balance': '0.00', 'usd_available': '0.00', 'usd_reserved': '0.00', 'usd_balance': '0.00', 'xau_available': '0.000000', 'xau_reserved': '0.000000', 'xau_balance': '0.000000', 'fee': '0.5000', 'fees': {'btc_cad': '0.5000', 'btc_usd': '0.5000', 'eth_btc': '0.2000', 'eth_cad': '0.5000'}}

Please report if everything is ok.


Add email alert attribute to checks closes #269

Following issue #269 - i've added an "Alert Email" field to the check edit screen - allows you to set a different email to receive the alert for this check.  I've added to the 'afterInsert' event this logic: check if the Check has an alert_email defined, if so, mail to there instead of what's defined in the config yaml file.

The would be useful for us too. @fzaninotto any change of getting this in for the next release ?


automatic config problem with paired samples numerically named with _samplenum

Not sure how to eloquently describe this problem. Imagine I have PE sequencing data for samples 1-12 named like this, unfortunately with no leading zeros.
sample_1_1.fastq.gz	sample_1_2.fastq.gz
sample_2_1.fastq.gz	sample_2_2.fastq.gz
sample_3_1.fastq.gz	sample_3_2.fastq.gz
sample_4_1.fastq.gz	sample_4_2.fastq.gz
sample_5_1.fastq.gz	sample_5_2.fastq.gz
sample_6_1.fastq.gz	sample_6_2.fastq.gz
sample_7_1.fastq.gz	sample_7_2.fastq.gz
sample_8_1.fastq.gz	sample_8_2.fastq.gz
sample_10_1.fastq.gz	sample_10_2.fastq.gz
sample_11_1.fastq.gz	sample_11_2.fastq.gz
sample_12_1.fastq.gz	sample_12_2.fastq.gz

When attempting automated configuration based on a CSV file using -w template, I get warnings that bcbio is adding minimal metadata for samples _1 and _2, and looking at the yaml file created, the files: list is incorrectly created.
I imagine it's something to do with how the template generation script is looking for _1.fastq.gz and _2.fastq.gz, but is getting confused by the _1 and _2 in the sample names themselves.
In any case, my workaround was to simply rename the files or symlink them without the _ between "sample" and the number. But it's probably not-that-edge-of-a-case potentially worth addressing, or making it at least obvious what's happening -- it took me a few minutes to figure out what the issue was.

I encountered exactly the same issue. I agree, that it seems to be related to the way bcbio looks for the PE files and then gets confused.

Sorry about that-- those file names are ambiguous so we have a hard time writing something to automatically detect the pairing, since we don't know if the _1 and _2 (or _3 and _4) apply to the first or second _1 or _2. For those particular samples we could guess because one goes higher than _4 but we wouldn't know if the experiment had < 5 samples. It's hard to think of a super good way to handle this case rather than renaming.

Makes sense. Might be worth a note at https://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#automated-sample-configuration

For FASTQ files, the template functionality will identify pairs using standard conventions (_1 and _2, including Illumina extensions like _R1), so use the base filename without these (/path/to/yourfile_R1.fastq => yourfile)

Hi,
thanks for the idea, do you want to make a pull request so it shows as your contribution?

On May 3, 2017, at 1:57 PM, Stephen Turner<EMAIL_ADDRESS>wrote:
Makes sense. Might be worth a note at https://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#automated-sample-configuration https://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#automated-sample-configuration
For FASTQ files, the template functionality will identify pairs using standard conventions (_1 and _2, including Illumina extensions like _R1), so use the base filename without these (/path/to/yourfile_R1.fastq => yourfile)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/chapmanb/bcbio-nextgen/issues/1919#issuecomment-298987310, or mute the thread https://github.com/notifications/unsubscribe-auth/ABi_HFBsjPmoQyWGYO0PD0BfMqJp_dOsks5r2L_ugaJpZM4NNA3R.

just made a PR.
On Thu, May 4, 2017 at 2:59 PM, Lorena Pantano<EMAIL_ADDRESS>wrote:

Hi,
thanks for the idea, do you want to make a pull request so it shows as
your contribution?

On May 3, 2017, at 1:57 PM, Stephen Turner<EMAIL_ADDRESS>wrote:
Makes sense. Might be worth a note at https://bcbio-nextgen.
readthedocs.io/en/latest/contents/configuration.html#
automated-sample-configuration <https://bcbio-nextgen.
readthedocs.io/en/latest/contents/configuration.html#
automated-sample-configuration>
For FASTQ files, the template functionality will identify pairs using
standard conventions (_1 and _2, including Illumina extensions like _R1),
so use the base filename without these (/path/to/yourfile_R1.fastq =>
yourfile)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <
https://github.com/chapmanb/bcbio-nextgen/issues/1919#
issuecomment-298987310>, or mute the thread <https://github.com/
notifications/unsubscribe-auth/ABi_HFBsjPmoQyWGYO0PD0BfMqJp_
dOsks5r2L_ugaJpZM4NNA3R>.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/chapmanb/bcbio-nextgen/issues/1919#issuecomment-299278252,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAcFLA8T66QCVrLZ_IKB-3hyiUUY1fcvks5r2iAkgaJpZM4NNA3R
.

Thanks! I'll close this.


Bugfix/400 error page

Description

Related issues

#365
Type of change

[x] Bug fix (non-breaking change which fixes an issue)

Changes Made

add error400 page
add unit and integration tests for error400 page

Testing

[x] Unit Tests
[x] Integration Tests

Mentions

Requires https://github.com/CSCfi/metadata-submitter/pull/240 (last commit is just for testing with that branch, and will be removed)

@hannyle I will remove it.


feat: add imagePullScrets

Description
This work has been done together with @gomezbc
The imagePullSecrets field allows you to specify secrets needed to pull Docker images from private repositories.
Why
In many organizations, container images are stored in private registries rather than public ones for security and compliance reasons. To pull these images, Kubernetes needs credentials, which are provided via imagePullSecrets. This code dynamically includes the necessary secrets for pulling images, based on the configuration provided in the Helm chart's values file.
Next Steps
Would it be interesting to add this implementation to other tractus-x charts?
Checklist
Please delete options that are not relevant.

[x] I have performed a self-review of my changes
[x] I have successfully tested my changes
[x] I have added comments in the default values.yaml file with helm-docs syntax ('# -- ') if relevant for installation
[x] I have commented my changes, particularly in hard-to-understand areas

@evegufy Thanks for the feedback.
I have added the changes you have requested concerning the comments in the values.yaml

@evegufy Thanks again for the feedback, I think the proposed changes are very good and have been implemented.

@evegufy You are totally right, my fault, I've gone overboard with the refactoring. Sorry.

@evegufy You are totally right, my fault, I've gone overboard with the refactoring. Sorry.

Hi @mgarciaLKS no worries, it's really awesome that you provided this feature 💯
As you asked under "Next Steps" :) this would be a nice feature also for the charts in the following repos:

https://github.com/eclipse-tractusx/policy-hub
https://github.com/eclipse-tractusx/ssi-credential-issuer
https://github.com/eclipse-tractusx/ssi-authority-schema-registry
If you'd like to look into it in the future.


Callbacks not working

If you try to set up a callback url, the Marathon responses are not read correctly:
Traceback (most recent call last):
  File "/usr/lib/python3.4/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "marathon_lb.py", line 1061, in wsgi_app
    processor.handle_event(json.loads(data))
  File "/usr/lib/python3.4/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
<IP_ADDRESS> - - [24/Dec/2015 00:52:41] "POST / HTTP/1.1" 500 59

Appears to be a Python3 issue I'm guessing per:

http://stackoverflow.com/questions/6862770/python-3-let-json-object-accept-bytes-or-let-urlopen-output-strings
https://github.com/boto/boto/pull/2778

Looks like json.loads expect a string but we're passing a bytes type.
but could be wrong as always :smile:

Got it somewhat working by decoding and re-encoding:
        processor.handle_event(json.loads(data.decode('utf-8')))
        start_response('200 OK', [('Content-Type', 'text/html')])

        return "Got it\n".encode('utf-8')

But it's spamming a ton of traceback (the haproxy reload is working though).


Support Flow Types

5.x.x has been converted to Typescript. If you're interested in helping restoring flow typed support, please comment here.

i'm interested. what is your current preference? libdefs decoupled and submitted to flow-typed or a more coupled approach internal to this project?

Absolutely coupled to the project. If they can’t be automatically generated then there is little hope of them ever being accurate. My hope is that there can be a build step that can generate them from the typescript definitions.

any preference on libraries to use to generate them?
found a few options, this seems mature enough to try: https://github.com/joarwilk/flowgen
a build step feels like a good place to put this process. the script will likely need to do more than just run the converter as there are likely a number of assumptions ts makes that flow does or visa versa, so some knowledge will be needed in writing more manual transformations. unclear though without actually trying to run it.
thoughts?
i can do a quick run of this and see how it looks in my project to see where it blows up?

Oh to be clear I tried to crack this nut using that tool plus some home rolled code and was not successful (although I think I got relatively close). Give it a go, maybe you can figure something out that I couldn't.

nice. any points / pain points you could roll into a sentence or two? i'ld be happy to look into this, it would provide me a lot of value. rmwc is a pleasure to use and would love to also have it provide my type safety in my code base too =]

Going to close out this issue. If anyone ever wants to take it one, feel free to reopen. But haven't heard the word flow in 6 months. It's marketshare is tanking while Typescripts is skyrocketing.


Upgrades gRPC. Releases frees-rpc 0.0.1.

Fixes https://github.com/frees-io/freestyle-rpc/issues/38
It also releases the first release (0.0.1) of freestyle-rpc giving some tiny instructions about how to use it.

Codecov Report

Merging #41 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #41   +/-   ##
=======================================
  Coverage   10.83%   10.83%
=======================================
  Files          20       20
  Lines         277      277
  Branches        1        1
=======================================
  Hits           30       30
  Misses        247      247

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba24696...81c563d. Read the comment docs.


make email obligatory in 'config_deploy.js' and remove ours as a default

The user's own email should be obligatory in 'config_deploy.js'. Currently, in 'scripts/submit_to_server.js' in line 10, ours is set as a default if no other email is specified. This should be removed.
Ideally, if no email is specified in 'config_deploy.js', there should be an alert box pop up on the first screen, which can be clicked away, so that everything will work eventually even without an email. The code 'submit_to_server.js' should then be amended to set 'contactEmail' to "No-email-address-provided"

Done here


PA9 and its 3-pin connector

Hi there,
I come here from your youtube channel and I want to make PCB for single hand keyboard. I was trying to copy your design idea. I found that you had a connector with Vcc, ground, and PA9, of which the usage is not explained in your video. What is the purpose  of this connector? Is it for debug for the PCB when you received from the manufacturer?
BTW, very nice video and cute cats XD
Thanks a lot

It could be used for anything, leds, another screen, or powering something else.


Userdata support

Expand the authentication system to allow for user data to be stored. Some form of ID is needed to allow the database to find the user. Maybe Google Auth, to get email address. See readme for info on theorized data structure.

[x]  Implement data structure
[x] Make API calls
[ ] Create secure authentication schema (and maybe integrate into current scheduling)
[ ] Integrate into Redis (perhaps on a new db?)
[ ] Utilize in applications

How about just localStorage? Most people only use their laptop, and don't
need their data synced across devices. Just an idea.
On Mon, Apr 11, 2016 at 5:57 PM, Saji Champlin<EMAIL_ADDRESS>wrote:

Expand the authentication system to allow for user data to be stored. Some
form of ID is needed to allow the database to find the user. Maybe Google
Auth, to get email address. See readme for info on theorized data structure.

Implement data structure
Make API calls
Create secure authentication schema (and maybe integrate into
current scheduling)
Integrate into Redis (perhaps on a new db?)
Utilize in applications

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/Foo-Bear/bearserver/issues/5

Fair point. How persistent is localStorage?
On Mon, Apr 11, 2016 at 5:58 PM, kincaidoneil<EMAIL_ADDRESS>wrote:

How about just localStorage? Most people only use their laptop, and don't
need their data synced across devices. Just an idea.
On Mon, Apr 11, 2016 at 5:57 PM, Saji Champlin<EMAIL_ADDRESS>wrote:

Expand the authentication system to allow for user data to be stored.
Some
form of ID is needed to allow the database to find the user. Maybe Google
Auth, to get email address. See readme for info on theorized data
structure.

Implement data structure
Make API calls
Create secure authentication schema (and maybe integrate into
current scheduling)
Integrate into Redis (perhaps on a new db?)
Utilize in applications

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/Foo-Bear/bearserver/issues/5

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/Foo-Bear/bearserver/issues/5#issuecomment-208603284

Local storage persists indefinitely, unless the user clears cookies
(Firefox) or clears cache (Chrome), depending upon the implementation. It
can store up to 10MB per domain, or something like that.
On Mon, Apr 11, 2016 at 6:08 PM, Saji Champlin<EMAIL_ADDRESS>wrote:

Fair point. How persistent is localStorage?
On Mon, Apr 11, 2016 at 5:58 PM, kincaidoneil<EMAIL_ADDRESS>wrote:

How about just localStorage? Most people only use their laptop, and don't
need their data synced across devices. Just an idea.
On Mon, Apr 11, 2016 at 5:57 PM, Saji Champlin<EMAIL_ADDRESS>wrote:

Expand the authentication system to allow for user data to be stored.
Some
form of ID is needed to allow the database to find the user. Maybe
Google
Auth, to get email address. See readme for info on theorized data
structure.

Implement data structure
Make API calls
Create secure authentication schema (and maybe integrate into
current scheduling)
Integrate into Redis (perhaps on a new db?)
Utilize in applications

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/Foo-Bear/bearserver/issues/5

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/Foo-Bear/bearserver/issues/5#issuecomment-208603284

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/Foo-Bear/bearserver/issues/5#issuecomment-208605978

But what about those people on IE6? :stuck_out_tongue:
I will look into it more, it would sure make life a lot easier for me.


Catch up on all new features up to v2.5.0 of DLL

v1.2.0

[x] #341 - https://github.com/DaveSkender/Stock.Indicators/pull/948 (v2.4.1)
[x] https://github.com/DaveSkender/Stock.Indicators.Python/pull/342(https://github.com/DaveSkender/Stock.Indicators.Python/pull/342) - https://github.com/DaveSkender/Stock.Indicators/pull/1081 (v2.4.11)
[x] #343 - (https://github.com/DaveSkender/Stock.Indicators/pull/1046) (v2.4.11)
[x] #344 - see https://github.com/DaveSkender/Stock.Indicators/pull/845 for explanation (v2.1.0)
[x] #337

[x] HtTrendline, SMI, StarcBand, TrueRange
[x] Add all nicknames of indicators
[x] 18a02f5 - https://github.com/DaveSkender/Stock.Indicators/pull/978 (v2.4.7)
[x] 511567a - https://github.com/DaveSkender/Stock.Indicators/pull/1030 (v2.4.10)

v1.3.0

[ ] McGinley Dynamic moving average - https://github.com/DaveSkender/Stock.Indicators/pull/875 (v2.2.0)

[ ] add configurable adjustment k-factor for McGinley Dynamic - https://github.com/DaveSkender/Stock.Indicators/pull/901 (v2.4.0)

[ ] ATR Trailing Stop - https://github.com/DaveSkender/Stock.Indicators/pull/891 (v2.3.0)
[ ] Chande Momentum Oscillator - https://github.com/DaveSkender/Stock.Indicators/pull/893 (v2.3.0)

@DaveSkender Ready to deploy v1.2.0 :) Do you have any lefts for this version?

@DaveSkender Ready to deploy v1.2.0 :) Do you have any lefts for this version?

I don't, but was just looking at the environment variables and don't see any of the tokens defined for either the test or production PyPI deployer, not do I see references to them in deploy.yml.  How did this even work last time we published without them?
... nevermind.  I just remembered they have a weird Trusted Publisher Management feature where you register on their site.  Unusual.

@DaveSkender I'd like to release it now as the version 1.2.1. How do you think?

@DaveSkender Please approve v1.2.1 when you are available 🚀

@DaveSkender
I'm thinking about not to support condense() for now. It might be more efficient for users to "condense" on their side.


Refactor GitHub Actions

Pull Request Test Coverage Report for Build 2ae12f02af0a1b733db2fc8bf2b4cb3dbf67fa04-PR-25

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build cf1d953ed0dab2a8641cf7b1228a5bee734c63b7:
0.0%

Covered Lines:
126

Relevant Lines:
126

💛  - Coveralls


[backport] Fix math function testing helper to support new style functions.

Backport of pull request #3603 from takagi/fix-test-math-function

LGTM!


x/build/gerrit: client is unwieldy

Using the Gerrit client, it feels like things are routinely more difficult than they should be. There are some points of friction that it would be nice to paper over:

Authentication: there are several methods of authentication. Gerrit tries to push you to git cookies auth, which isn't great for bots/logic that just talks to the REST API, and if you want to do basic auth you have to awkwardly parse out the username/password out of the gitcookies auth UI. NewClient should default to BasicAuth, you shouldn't have to figure out which method of authentication is appropriate and how to get tokens

Posting comments: You have to know 1) Gerrit treats line by line comments differently from on-the-totality-of-the-review comments and 2) which patch set you want to leave a comment on. If you are writing a bot all you need/want to do is post a comment on the latest patch set. This should be simpler to do in the Gerrit client.

Query parameters are annoying: Gerrit lets you ask for more data by specifying o=MESSAGES&o=CURRENT_COMMIT etc, in Go this turns into
QueryChangesOpt{
    Fields: []string{"MESSAGES", "CURRENT_COMMIT"},
}

These structs make it sort of difficult to go back and forth between the API description on https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#list-changes and the abstraction that hides this in the client. It might be easier as just a url.Values{}.
I wouldn't underestimate how much of Github/Slack's success is due to API's that make them easy to interact with, it's not fair to ask Gerrit to get to that level but it would probably help people create interesting things around Gerrit if it was easier to get started with the API's and the clients.

Inconsistent verbs.

The method names match the API names on purpose. https://godoc.org/github.com/andygrunwald/go-gerrit does the same.
I have no plans of redoing our Gerrit client to flesh it out or change its feel. Its goal is to be the minimum we need for the Go build system.

Sure, I guess the cost is it's going to be harder at the margin to write code to interact with our code review tool.


Fixed useNativeDriver warning issue

useNativeDriver must be specified explicitly now, so added it to props and default value will be false.

@lhandel Can we please get this merged?

Bump to merge this PR :100:

Can we get this merged, please?

Please merge @lhandel

Bump for merge!

Please merge @lhandel


Make RAND_METHOD opaque

See some discussion in #3797

Its it worth considering making a no-op that does nothing?

Or maybe always return an error.  Not sure.

Side note: shouldn't this be labeled 1.2.0? That would clearly mark it for the next major release.
(I thought I had created that label, why did it disappear?)

closing this as #10017 has more discussion.


Added italics to project name

Wrong place! This is the about page data that only appears by default if we don't add custom about page data. See libcrowds frontend settings.


Consider ways to improve fitness

Recent research points to inadequacy of sharpe (alone) as a fitness measure. Consider weighted fitness using tail ratio, benchmark alpha, kurtosis etc.

IMO I think this should be left open as part of the users algorithm. It entirely depends on what they are trying to achieve, which means they should also decide how it is scored. We can of course include some defaults. For example, someone looking to build a long term strategy may want to measure both max drawdown and also Sharpe. But someone else may not care about risk so much and just wants to measure dairy profits. We could perhaps create an IOptimizedAlgorithm interface which people apply to their algo, and then they fill in the FitnessEvaluation themselves.

The default fitness is based on sharpe ratio. There is another fitness measure provided based on annual returns. There is also a configuredfitness class that allows you to optimize based on any other statistic field from the main output.
I'm not convinced that a fitness measure is a concern of an algorithm: I believe at the very least a benchmark is going to be appropriate to a family of resemblances.
If you wish to define your own algorithm interface pattern for integrating with the config then feel free. You could also define your own benchmark and configure the optimizer towards this. However I'm not aware of strong supporting evidence that point to the inadequacy of Sharpe ratio. This is one of the project's main areas of research interest. If you are aware of any pertinent research papers, please post here.
On 10 April 2017 01:26:49 BST, Worthy7<EMAIL_ADDRESS>wrote:
IMO I think this should be left open as part of the users algorithm. It entirely depends on what they are trying to achieve, which means they should also decide how it is scored. We can of course include some defaults. For example, someone looking to build a long term strategy may want to measure both max drawdown and also Sharpe. But someone else may not care about risk so much and just wants to measure dairy profits. We could perhaps create an IOptimizedAlgorithm interface which people apply to their algo, and then they fill in the FitnessEvaluation themselves.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/jameschch/LeanOptimization/issues/4#issuecomment-292824440, or mute the threadhttps://github.com/notifications/unsubscribe-auth/APvtWCkPdgEP9NFIUtKbiiNk4TdwbhDEks5ruXdJgaJpZM4LqU0V.

This is the paper that prompted this issue:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2745220

There has been some interest in a blended fitness, so for instance 50% win rate and 50% alpha. I suspect the blend ratio this would just be another hyperparameter liable to overfitting.

Fair enough, it seems you've done a bit more research than I.

Well, this is just the tip if the iceberg. Please go ahead and try some options with the ConfiguredFitness. If there is a missing statistical measure it would be valuable to Lean in general.

You're most likely correct, I only started using QC about 2 weeks ago. Still learning, but eager!
I hope I can contribute to this project at some point. I loved my Machine Learning course back in university, but I feel first some performance optimizations might be a good place to start though.

I mentioned this on the forum, but I'll mention it here too. How about just net profit %?
I know it sounds simple, but when this is done using walk forward, then that somewhat takes care of the issues that thing's like "max drawdown" are supposed to deal with. It's literally just raw money return...  Unless I'm not understanding the Walk Forward behavior properly... ( I read the second half of the paper btw, I see the point, but I'll need to read it again to understand it properly)

Read it all again, in detail. Someone also summarized it online:
https://optimal.quora.com/Trading-algorithms-Comparing-Backtest-and-Out-of-Sample-Performance-using-Machine-Learning
Why don't we:

Somehow restrict the Sharpe to only the last year of data. (By either, suggesting it to the user, or overwriting the dates set in their data.)
Use the tail ratio as the default fitness function... It seems to be much more correlated.

Tbh, the paper DOES seem to suggest a fairly strong issue with Sharpe, especially since everyone is backtesting as far as they humanly can. Rather than depth, people should be going for breadth (across more markets/pairs).
10 different pairs over the last 1 year is better than 1 pair over 10 years, and, arguably, it could be better than 10 pairs over 10 years.

These is already a fitness based on return provided in the project. The flaws of this measure are what gave rise to Sharpe ratio.
On 26 April 2017 07:06:41 BST, Worthy7<EMAIL_ADDRESS>wrote:
I mentioned this on the forum, but I'll mention it here too. How about just net profit %?
I know it sounds simple, but when this is done using walk forward, then that somewhat takes care of the issues that thing's like "max drawdown" are supposed to deal with. It's literally just raw money return... Unless I'm not understanding the Walk Forward behavior properly... ( I read the second half of the paper btw, I see the point, but I'll need to read it again to understand it properly)
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHubhttps://github.com/jameschch/LeanOptimization/issues/4#issuecomment-297247515, or mute the threadhttps://github.com/notifications/unsubscribe-auth/APvtWIxI5ZzP-131fQayxNp4OYnlMrQNks5rzt7xgaJpZM4LqU0V.

Yea, I realize that now. Ignore that comment  :-)

I like the idea of a blended fitness. However I think without some theoretical basis, the blend % will just be another overfit parameter.

What do you mean by blend %? you mean the 0.65 & 0.7 figures? They are from the paper.

In that case this would be a good candidate for fitness. Let me look closer at the paper.

Just to clarify a little bit (I'm looking to work on the code soon, so rereading the article a bit):

Basically, for any kind of backtesting it suggests that Tail-Ratio is a better predictor of REAL WORLD Sharpe. That's interesting.

If we are going to use Sharpe, it suggests that the latest year of backtest data to be used and nothing more. Sharpe has 'recency'.

So basically, I want to implement the tailratio statistic first. Perhaps it should even be a part of Lean.
At the very minimum, using tailratio for optimizing seems to be a better indicator of how the algorithm would perform in the real world, if performance is based on Sharpe.
I think my idea for blending can be ignored for now, It's situational in that it requires the use of only the last year of data. We would somehow have to recognize and check for this and it seems it everything can just be improved by using tailratio instead.

Sounds like an excellent idea to try to get tail ratio integrated into lean.
My understanding is that there is a recency bias for Sharpe but I imagine you will find a recency bias for many measures due to capturing the current regime.

Yea but think about it, the tail-ratio should be pretty robust. It is just daily profits? How raw does it get :D
Key word being "Daily" not "Over time"!
I have no idea how to get a Daily Profits statistic out of lean. Any ideas?


Added optional parameter for bibliography style

This adds the feature to specify the citation / bibliography style. IEEE is good but some prefer another style especially in long papers.

Hey there,
Thanks for the contribution 🙏🏻
I added your suggestion to my current PR in the typst repo. The PR contains other improvements and fixes. Feel free to take a look here https://github.com/typst/packages/pull/778


Add I/O and obsop for SSS (resolve #134)

Brief description
Add I/O and obsop for SSS (resolve #134 )
Features added

Add module support/io/m_h5io.f90 for convenient IO of HDF5 files.
Add module src/obs/read_smap.f90 for read in L2/L3 SSS JPL SMAP retrievals.
Add obsop src/obs/obsop_sss.f90 for SSS simulation.
Add utils/obs_proc/level-3/get_l3_smap.py for getting L3 SMAP SSS.
Update CMAKE to include HDF5 dependency.

Bugs fixed
Test changes

Add one test for getting L3 SMAP SSS under the action get_data.

Documentation changes
N/A
Does this create a breaking change?
N/A

Hi @sanAkel, do you happen to have time to review this PR? I haven't seen any of your review yet. I have some other PRs to push depending on this PR. Thanks!

Hi @sanAkel, do you happen to have time to review this PR? I haven't seen any of your review yet. I have some other PRs to push depending on this PR. Thanks!

@cd10kfsu I plan to review it (not feeling well, when I am better, will do).


Hide "+ Add missing language"?

While I can see this rather big button being useful in some cases, I have had requests from our users to hide it since it's just taking up valuable space. If a site only has 2 langs and one is currently filled out, for example, there's two buttons doing the same thing 🤷♂
An option to hide this would be great! Or if someone could help me how to target these with css to hide manually?
🙏🏻

Version 1.8.0 has a configuration option buttonAddAll which can be set to false to hide the "Add missing languages" button
import {defineConfig} from 'sanity'
import {internationalizedArray} from 'sanity-plugin-internationalized-array'

 export const defineConfig({
  // ...
  plugins: [
    internationalizedArray({
      // ...other config
      buttonAddAll: false // default true
    })
  ]
})

I tried updating to 1.8.0 but my inputs suddenly looked very different. See attached image.
Any idea on what might cause this?

@oskarengstrom That button should only appear when you have array keys that aren't found in the language configuration. Could you paste your schema for that "Global" field?

Well, I never changed any configurations after bumping version 🤷♂
Sure thing, here's the schema:
export default {
  name: 'leadQuizModalPage',
  type: 'document',
  title: 'Lead Quiz Modal Page',
  icon: BookIcon,
  fields: [
    {
      name: 'global',
      type: 'object',
      title: 'Global',
      fields: [
        {
          name: 'closeLabel',
          type: 'internationalizedArrayString',
          title: 'Close Label',
        },
        {
          name: 'title',
          type: 'internationalizedArrayString',
          title: 'Title',
        },
      ],
      options: {
        collapsible: true,
        collapsed: true,
      },
    },

Thanks, and what is your languages config in sanity.config.ts?

    internationalizedArray({
      languages: [
        {id: 'en', title: 'English'},
        {id: 'de', title: 'German'},
      ],
      defaultLanguages: ['en'],
      fieldTypes: ['string', 'text'],
    }),

Ah thank for that. Think I've found the cause of the issue. Will update this thread with a solution soon.


Add output directory to glfw_generator

adds --out= which defaults to 'generated'
cleanup of some headerfiles
update README with formatting

LGTM


Robin karp algorithm in python

Is your feature request related to a problem? Please describe.
There is no pattern matching Algorithm in python
Describe the solution you'd like
To implement pattern matching algorithm in python
Do you want to work on it
I want to work on this issue in python

I would like to contribute to the above problem in java as I am a participant of GSSOC'21. Kindly assign this to me.


JupyterLab automatically select default kernel

Start with the base kernel, don't show the popup on each notebook.

Add a parsing section to nbconvert;
"kernelspec": {
    "display_name": "mantaray",
    "language": "python",
    "name": "mantaray"
},


Cherry pick fix for macOS build

🦟 Bug fix
Cherry-pick #1599 forward to ign-gazebo6
Summary
This fixes the build with new versions of protobuf in use on macOS. I tried merging it forward, but there were tons of conflicts, so I'm cherry-picking it to more quickly fix CI.
Checklist

[X] Signed all commits for DCO
[ ] Added tests
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] Consider updating Python bindings (if the library has them)
[ ] codecheck passed (See contributing)
[ ] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Rebase-and-merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

can we make a forward port ? why just this commit ?

there were a lot of conflicts that I didn't know how to properly resolve. I've started resolving some of them in https://github.com/gazebosim/gz-sim/pull/1602


[incubator/kube-spot-termination-notice-handler] change repo

Because the original app maintainer appears to not be available anymore,
a fork of the project (not of this chart) has been created, incorporating new features and a simpler
build process (now is an auto-build project in Docker Hub, not a
locally executed makefile).

Any chance of getting this merged in soon? 😄

/ok-to-test


System.err: java.lang.NoSuchMethodError: No static method aspectOf()Lcom/jidu/lib/data_engine/FragmentViewAspect; in class Lcom/jidu/lib/data_engine/FragmentViewAspect; or its super classes

@DengFX2010105421 can you point to a sample project to see this issue or able to reproduce in sample? Being able to see code would be helpful in triaging

This error is unrelated to the ibotta aspectj plugin. The error was thrown on the user's code. @DengFX2010105421 you probably need to invalidate the gradle cache and re-build your app.
@jdvp I recommend closing out this issue.

This error is unrelated to the ibotta aspectj plugin. The error was thrown on the user's code. @DengFX2010105421 you probably need to invalidate the gradle cache and re-build your app.
@jdvp I recommend closing out this issue.

Thanks, community, for helping out with this one! 👍


drivers: serial: stm32 uart defines dma slot with macro

The STM32_DMA_SLOT macro from include/drivers/dma/dma_stm32.h
must be used here, especially for dma of type v2bis.
In this case, the dma-cell is not defined and slot is null.
This will fix compilation error when building tests/drivers/uart/uart_async_api for nucleo_l152re or stm32f3_disco
or any device with DMA of type V2bis.
Signed-off-by: Francois Ramu<EMAIL_ADDRESS>

This PR is already in, thanks to the merge of PR   #36402 "tests: drivers: more testcases running on the nucleo_l152re board"


grpc error when trying to import keys with slashing_protection

Hi,
I'm trying to import a validator to a new node, also want to move using this great project (BTW, thanks a lot for it!)
Completed guidelines as in https://eth-docker.net/Support/Moving
Exported the keystore and slashing_protection json using prysm commands (was using native dockers in old node).
While running ./ethd keys import
.
.
Found slashing protection import file /validator_keys/slashing_protection.json for ****
It will be imported
Unexpected return code 429. Result: {"message":"grpc: received message larger than max (22150665 vs. 4194304)","code":429}
My slashing_protection  file is indeed ~27MB
How can I go around it ?
Much appreciated
Ido

Hey @shamoya 👋
it looks like prysm supports max 4MB. This can be increased by setting --grpc-max-msg-size value CLI option to a higher value in your case something like  23000000. You should be able to pass this to VC_EXTRAS in your .env file, like this VC_EXTRAS=--grpc-max-msg-size value 23000000 and then run ./ethd up to apply the changes.
https://github.com/eth-educators/eth-docker/blob/efcba2784c49f53cbb81b8acb7e240bfb5ee4321/default.env#L97-L98

Thank you @nflaig for the quick reply.
I'll give it a try. closing this anyway


Use UserAttribute.ExtraByProvider to populate impersonation role rules

Followup from https://github.com/rancher/rancher/issues/34671
Depends on https://github.com/rancher/rancher/pull/33268. That PR will introduce a unified way to get a user's extra attributes from their auth provider, which we will be able to leverage to simplify how we generate the clusterrole rules related to user extras.

SURE-4361

Moving to test, but I think this can just be tested via https://github.com/rancher/rancher/issues/36096

Tested via  https://github.com/rancher/rancher/issues/36096#issuecomment-1103222428


only populate the necessary repo

Instead of populating both the READERS and WRITERS repo on any operation, only the relevant repo will be populated. For a very complex schema I noticed a 5-10% speed up in writing and a 10-20% speed up in reading.

Thanks


NoWhitespaceInBlankLineFixer - Fix more cases

Fix cases like this as well (not trailing white space on line 1)
<?php

$a = 1;

appveyor is broken, dunno how to fix (ping @keradus ;) )

Thank you @SpacePossum.


[RFR] Fix Graphcool demo

@fzaninotto There are still some warnings though, linked to the new/updated features of next. Do you want me to address them in this PR too ?

Oh yes please


[Feature] #30 Export to Excel file

I've done some basic implementation which supports downloading to excel files.
Of course, a lot of improvements can be added, but, IMO, the core is complete.
N.B. Added endpoint raises 401 error because there is no authentication on rest api endpoints, and downloaded spreadsheet contain sensitive information.
Review request: @webknjaz

codestyle fixed in 9f6a6bf
securing filename before making response in 873459f
raising 404 Not found when there is no event found in 6217a21

As for request type^ what should it be? POST?

Nevermind about the method. It's more like a philosophical problem.

Removed GDG-Ukraine from imports 9ed297f
Added event id and event date to xls filename bd56fba


[Mobile] Saving the text to device storage as file (ex. txt)

this is actually not an issue, i just started learning flutter a view weeks ago.
i created an Android App for my project.
so, i've created an editor page succesfully, and i want to save the data that have been typed on the editor to my device storage and after it's saved, i can access the file from my device file manager.
the file must be saved as extension file (ex: txt files).
i hope i got a detail explaination for this,..
and hope somebody could help me, i really need it!
sorry for my bad english.

To access the text, you must assign a QuillController to the QuillEditor:
  late QuillController _controller;
  @override
  void initState() {
    super.initState();
    _controller = QuillController.basic();
  }

  @override
  Widget build(BuildContext context) {
    return QuillEditor(
        controller: _controller,
        /* ... */
    );
  }

You can then access the text using _controller:
final text = _controller.document.toPlainText();

Look here to save text to disk and read it.

thanks for the advice..
i'll try the code from path_provider library you'd mention..
if it's if not troublesome for you i'll ask some advice if some problem hapens later!
thank you very much..

haii... @friebetill
i've tried your advice before and follow the tutorial about how to read and write data to file using path_provider Library.
but after all code i'd build, my save button did not work at all..
i try a lot of method to save data to a file in youtube but didn;'t work either..
im really confuse..
can you give me a code as example to implement flutter_quill and path_provider, then save the data i've type to a file in local directory with a button..
i really need your help..
thank you...


US 01.03.01

User Story
As an organizer, I want to send notifications to all attendees through the app
Acceptance Criteria

Given

Inside the specific event page, there will be a tab/button that will bring a popup fragment to create a notification

Action

[ ] When the tab/button is clicked it should display a popup fragment where the organizer can write a text notification
[ ] In the same popup, there will be an option to cancel or send the notification.

[ ] If "send" is clicked, another popup should confirm the notification text and ask the organizer to reconfirm.
[ ] If "cancel" is clicked, another popup should confirm the discard action and, upon reconfirmation, discard the notification and return to the event page.

Then

If the organizer reconfirms the send action, the notification should be sent to all attendees, If the organizer reconfirms the cancel action, the notification is discarded.

Additional components

Storyboard demonstrating the feature
Multi-User Interaction
Firebase Integration

User Story
As an organizer, I want to send notifications to all attendees through the app
Acceptance Criteria

Given

Inside the specific event page, there will be a tab/button that will bring a popup fragment to create a notification

Action

[ ] When the tab/button is clicked it should display a fragment with all the previously sent notifications if any.
[ ] In the notification fragment, there is a button to create a notification. If clicked, popup will appear where the organizer can write a text notification.
[ ] In the same popup, there will be an option to cancel or send the notification.

[ ] If "hold to send" is long-clicked, the notification will be sent to all attendees
[ ] If "hold to cancel" is long-clicked, the organizer will return to the event page.

Then

If the organizer reconfirms the send action, the notification should be sent to all attendees, If the organizer reconfirms the cancel action, the notification is discarded.

Additional components

Storyboard demonstrating the feature
Multi-User Interaction
Firebase Integration


[Question]: "How to define the most strict configuration possible for  Shellcheck  ?

What is your question?
I’m collaborating in a opensource project :
https://bitbucket.org/AliasAPI/damnp/src/master/etc/shellcheck/run_shellcheck.sh
And i am finding trouble configuring Shellcheck  in the most sensitive way possible, so it would detected every generally undesired line of code. If anyone has any tips, it would be very welcome.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Out of the box shellcheck will perform ALL checks, which is as secure as it can be. There is no additional configuration that you can add in order to extend it, the only thing you can do is turn off checks/features using the -e flag (which I can see you are already doing).
You could look at something like bats (https://github.com/sstephenson/bats) which might add a level of testing to the code but beyond that there isn't much else for static analysis for bash.

I would also recommend looking at another little tool/demo I wrote.
https://github.com/DevelopersToolbox/bash-snippets/blob/master/src/strict-mode/strict-mode.sh
This allows you to make sure your scripts are as strict as possible.


Skip "claimed CD" creation webhook for DR

We have a webhook to disallow creating a ClusterDeployment with the
ClusterClaim already set. This is a problem for disaster recovery. Add a
granny switch to disable that specific check via the following label:
"hive.openshift.io/disable-creation-webhook-for-dr": "true"
HIVE-1791
Conflicts:
pkg/constants/constants.go
...for IBMCloud again.

/assign @abutcher

/lgtm