Define scope of needed Best Practices documents

It is necessary to describe what Best Practices documents need to be written,
especially if more than one.

Original issue reported on code.google.com by morris.bob on 4 Mar 2012 at 8:28

Blocking: #14, #14

Are there DwC examples to study? Note that AC now contains DwC.


Protein crotonylation and protein butyrylation

Hi,
I would need a series of terms for a number of protein acyltransferase activities. It looks like protein acetylation is not the only protein acylation and that other acylation modifications are present on lysines
I have a mixed feeling concerning creating specific terms for histones. We did that in the past, but on the other hand, histones are proteins and most enzymes have non-histone substrates
Thanks
Sylvain

Name: Protein crotonyltransferase activity
Category: Function
Definition: Catalysis of the reaction: crotonyl-CoA + lysine in peptide = CoA + N-crotonyl-lysine-peptide.
Child of GO:0016410 N-acyltransferase activity
PMID:25818647

Name: Protein butyryltransferase activity
Category: Function
Definition: Catalysis of the reaction: butyryl-CoA + lysine in peptide = CoA + N-butyryl-lysine-peptide.
Child of GO:0016410 N-acyltransferase activity
PMID:27105113

Name: peptidyl-lysine crotonylation
Category: Process
Definition: The crotonylation of peptidyl-lysine.
child of GO:0043543 protein acylation
PMID:25818647

Name: peptidyl-lysine butyrylation
Category: Process
Definition: The butyrylation of peptidyl-lysine.
child of GO:0043543 protein acylation
PMID:27105113
We migh also create specific terms for histone crotonyltransferase activity and histone butyryltransferase activity, but I am unsure that we should systematically create specific terms for histone modifications since in most cases, enzymes also modify non-histone proteins

Name: histone crotonyltransferase activity
Definition: Catalysis of the reaction: crotonyl-CoA + histone = CoA + crotonyl-histone.
child of Protein crotonyltransferase activity
PMID:25818647

Name: histone butyryltransferase activity
Definition: Catalysis of the reaction: butyryl-CoA + histone = CoA + butyryl-histone.
child of Protein butyryltransferase activity
PMID:27105113

Hello @sylvainpoux
I created the following terms:
[Term]
+id: GO:0140064
+name: peptide crotonyltransferase activity
+namespace: molecular_function
+def: "Catalysis of the reaction: crotonyl-CoA + lysine in peptide = CoA + N-crotonyl-lysine-peptide." [PMID:25818647]
+synonym: "protein crotonyltransferase activity" RELATED []
+is_a: GO:0008080 ! N-acetyltransferase activity
+created_by: pg
+creation_date: 2017-07-20T16:40:57Z
+
+[Term]
+id: GO:0140065
+name: peptide butyryltransferase activity
+namespace: molecular_function
+def: "Catalysis of the reaction: butyryl-CoA + lysine in peptide = CoA + N-butyryl-lysine-peptide." [PMID:27105113]
+synonym: "protein butyryltransferase activity" RELATED []
+is_a: GO:0008080 ! N-acetyltransferase activity
+created_by: pg
+creation_date: 2017-07-20T16:46:49Z
+
+[Term]
+id: GO:0140066
+name: peptidyl-lysine crotonylation
+namespace: biological_process
+def: "The crotonylation of a lysine residue in a protein. Crotonyl is the univalent radical CH3-CH=CH-CO- derived from crotonic acid." [PMID:25818647, Wikipedia:crotonyl]
+is_a: GO:0043543 ! protein acylation
+created_by: pg
+creation_date: 2017-07-20T16:48:58Z
+
+[Term]
+id: GO:0140067
+name: peptidyl-lysine butyrylation
+def: "The butyrylation of a lysine residue in a protein. Butyryl is the univalent radical C3H7COO- derived from butyric acid." [PMID:27105113, Wikipedia:butyryl]
+is_a: GO:0043543 ! protein acylation
+created_by: pg
+creation_date: 2017-07-20T16:52:19Z
+
+[Term]
+id: GO:0140068
+name: histone crotonyltransferase activity
+namespace: molecular_function
+def: "Catalysis of the reaction: crotonyl-CoA + histone = CoA + crotonyl-histone." [PMID:25818647]
+is_a: GO:0140064 ! peptide crotonyltransferase activity
+created_by: pg
+creation_date: 2017-07-20T16:55:16Z
+
+[Term]
+id: GO:0140069
+name: histone butyryltransferase activity
+namespace: molecular_function
+def: "Catalysis of the reaction: butyryl-CoA + histone = CoA + butyryl-histone." [PMID:27105113]
+is_a: GO:0140065 ! peptide butyryltransferase activity
+created_by: pg
+creation_date: 2017-07-20T16:56:33Z
+
A few comments:

I modified the name of 'protein crotonyltransferase activity' and 'protein butyryltransferase activity' to peptide to match the neighboring terms. I added your term names as synonyms.

Do you think we should create a general parent 'peptide N-acyltransferase activity' to group all these related modifications ?

Thanks, Pascale


Update IAM guide for users installing Snowplow Kinesis

I guess this warrants a whole new guide?
Notes from @pkallos' setup guide for Kinesis S3 sink:

The AWS permissions required to run this app are:
CloudWatch -- I tested with full access
DynamoDB -- I tested with full access. Needs to be able to create table, write to that table. The Kinesis library does checkpointing by creating a Table called "application-name" and periodically updating a key that stores the checkpoint.
Kinesis -- I tested with read/write access to a specific stream. Needs at least permission to read from the stream specified in the config.
S3 -- I tested with read/write access to a specific S3 bucket. At least permission to write to the configured S3 bucket.

https://github.com/snowplow/snowplow/wiki/Setup-IAM-permissions-for-users-installing-Snowplow

See also: #1490

This is done, closing as it's not for CHANGELOG.


Avoid zoom caused by procedures on open image

I am trying to avoid zoom which caused by OSD procedures when open an image. Hence i am trying to set viewer property with preserveViewport = true . It was works as expected when i am not set the width on viewer.open() properties at the parameter. It was like follow :
    self.viewer.open({
	tileSource:{
	    width: data.width, height: data.height,
	    tileOverlap: 2,  tileSize: 250,
	    getTileUrl: function( level, x, y ){
		return "/drawing/tiles/"+page_number+'/'+level+"_"+x+"_"+y;
	    }
	}
    });

But if i add width properties at the parameter as follow, sometimes it will add at least 3 times of zoom event ( 3 step zooming until reach the last zoom level ) :
    self.viewer.open({
	width: data.width,
	tileSource:{
	    width: data.width, height: data.height,
	    tileOverlap: 2,
	    tileSize: 250,
	    getTileUrl: function( level, x, y ){
		return "/drawing/tiles/"+page_number+'/'+level+"_"+x+"_"+y;
	    }
	}
    });

Is it possible to avoid zoom event in any other way after viewer.open()? Although i am not sure what causing the slight zooming after open an image. Thanks in advance.

I imagine the zoom is just changing the viewport from its default to something that matches your new coordinates. I'm not sure why there would be three of them, though.
Anyway, probably your best bet is to detect this circumstance and suppress handling of the events, just like the other scenario we discussed. Do the events happen before or after the open event is fired?

Sorry, seem it was my mistakes. I have Viewport.goHome() which i was thought it will not producing zoom in animation. But it does give me 3 step zooming for animation.


fix/345: clean up redundant code

isVowel应该是单数合适一些
continue 可以减少if分支判断

Codecov Report

Merging #127 (2450cf4) into master (2482928) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #127      +/-   ##
==========================================
- Coverage   71.82%   71.81%   -0.01%
==========================================
  Files         673      673
  Lines       11953    11950       -3
==========================================
- Hits         8585     8582       -3
  Misses       3056     3056
  Partials      312      312

Impacted Files
Coverage Δ

...els-of-a-String/345. Reverse Vowels of a String.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2482928...2450cf4. Read the comment docs.


Decide which methods of the datatype each pair should test

For the datatype chosen in #16 and pairs chosen in #11, decide which methods in the datatype each pair should test. The division of work should be roughly even.
Deliverable: create and assign issues to each pair/triplet in the group that specifies the methods to test. The tests shoud:
1. Measure the runtime and, if possible, coverage of the existing back-to-back regression tests for this method.
2. Write unit tests clamping the Java implementation of this method to its present (validated) behaviour. Measure their runtime at the same coverage as the back-to-back regression tests for this method.
3. Where possible, add additional unit tests to improve coverage.

There is also a note in instructions that "In order to clamp the Java code before editing it, the goal is to obtain high coverage. Consider the finest coverage metric you can automatically obtain. In Sprint 3, we will then profile and modify these data types."

[ ] __setattr_null
[ ] List constructor
[ ] setitem
[ ] getitem
[ ] append
[ ] extend
[ ] remove
[ ] reverse
[ ] count
[ ] contains
[ ] sort
[ ] pop
[ ] copy
[ ] index
[ ] lt
[ ] le
[ ] gt
[ ] ge
[ ] eq
[ ] reversed

Division is as follows:
-- PAIR 1 --
public java.lang.Object toJava()
public int hashCode()
public org.python.Object hash()
public boolean isHashable()
public List()
public List(java.util.List<org.python.Object> list)
public List(org.python.Object[] args, java.util.Map<java.lang.String, org.python.Object> kwargs)
public org.python.Object new()
public org.python.Object init()
public org.python.Object iadd(org.python.Object other)
public org.python.Object bool()
public org.python.types.Str repr()
public org.python.types.Str format()
public org.python.Object lt(org.python.Object other)
public org.python.Object le(org.python.Object other)
public org.python.Object eq(org.python.Object other)
public org.python.Object gt(org.python.Object other)
public org.python.Object ge(org.python.Object other)
public boolean __setattr_null(java.lang.String name, org.python.Object value)
public org.python.types.Int len()
public org.python.Object getitem(org.python.Object index)
-- PAIR 2 --
public void setitem(org.python.Object index, org.python.Object value)
public void delitem(org.python.Object index)
public org.python.Object iter()
public org.python.Object reversed()
public org.python.Object contains(org.python.Object item)
public org.python.Object add(org.python.Object other)
public org.python.Object mul(org.python.Object other)
public org.python.Object imul(org.python.Object other)
public org.python.Object rmul(org.python.Object other)
public org.python.Object append(org.python.Object item)
public org.python.Object clear()
public org.python.Object copy()
public org.python.Object count(org.python.Object other)
public org.python.Object extend(org.python.Object other)
public org.python.Object index(org.python.Object item, org.python.Object start, org.python.Object end)
public org.python.Object insert(org.python.Object index, org.python.Object item)
public org.python.Object pop(org.python.Object item)
public org.python.Object remove(org.python.Object item)
public org.python.Object reverse()
public org.python.Object sort(final org.python.Object key, org.python.Object reverse)
public int compare(org.python.Object o1, org.python.Object o2)
public org.python.Object round(org.python.Object ndigits)


Invalid states not working on some componentes

Hi,
On AutoComplete, Calendar, Chips, inputSwitch, inputTest in the documentation it says to use p-invalid for the invalid state, but that's not working (as it can be noticed in the documentation itself)
For Dropdown it says to use p-invalid but in the example are tailwind classes:  !border !border-red-300 !dark:border-red-400 w-full md:w-14rem
On some components suggested approach is to use p-invalid while on others (that have proper working examples) it's being done with multiple classes. What should be the proper approach?
As we have themes I would expect that the invalid states are part of the theme and they don't need to be custom-written every time. Maybe there can be an example of how to write our own invalid style

Hey @slaven-ii 👋
I agree that the documentation is a bit confusing, but I think the current approach is like this Extracting classes with @apply . I'm currently using that, for example for p-invalid or router-link-active.

Hi, yes, the invalid demos are confusing. We will make them like the disabled, so in the preset, you will be able to give your classes for invalid. This needs changes in the PrimeVue core (PT). We will remove the invalid demos for now. We plan to add this as soon as possible.

@alfonsovgs Thanks, with the plans that @atakantepe explained this does look like a good solution for now. Thanks both


The duration display the thresold duration instead of how long the app has been blocked

In the v1.x, the duration logs in the console was the duration of how long the main thread has been blocked. Starting with the 2.0, the duration is always displaying the threshold duration.
That's a sad regression, it was a really interesting feature. Do you think to bring it back?

#24

Any news about this? 😞

Hi Tom. It shouldn't be hard to concat version 1.x and 2.x together. However, using solution from 1.x (based on NSRunLoop) it is not possible to find out the exact code which block the main thread. Also I wouldn't recommend using watchdog in production to find proper time of blocking main thread.


Fix File.extname for filenames like "a.yml", and make the method easier to understand

Fixes #6215
Alternative to #6216, which probably works, but please try to explain me in words what this means:
if dot_index && dot_index != filename.size - 1 && dot_index > ((filename.rindex(SEPARATOR) || -1) + 1)

The regression was probably introduced because the code became hard to understand, practically unreadable. It's better to have small concise checks with comments, so that anyone can quickly understand what the code does.

Actually, nevermind. It should be done like in Go: https://golang.org/src/path/filepath/path.go?s=6131:6159#L207

(But I won't send a PR, so someone should do that. Probably going byte per byte, in reverse, over the slice, should work, but it's a boring thing to work on for me)

@asterite This is actually already implemented in #5635 as Path#extension. This could be backported as bugfix for File.extname, so it can go into 0.25.1.


Week 4 libraries/comments

Added comments both for backend and frontend logic
The comments were added following Google styleguide specification both for Java and Javascript. Even though I think I can be even more specific and detailed on commenting, I think this is enough.
Solves some suggestions @livvielin made on #9 regarding to pagination and the specifications I made there.
Let me know if you have any comments.
Note: Please review and approve #11

Thanks for writing a bunch of comments!

You're welcome 😄


AutoPlay

Hi,
I wonder how I can remove autoplay and add a thumbnail to the video player?
Best regards

You have to fill in the issue template fully. Also make separate issues for different features.


added blue color to the scrollbars

If you don't like the color, please mention the color you want.

Testing Plan:
GIFs or Screenshots:

Hi @namannimmo10, there's some reading that might be good to do while attempting to tackle this issue.

In order to accomplish what Rishi mentioned (going pretty far back in the commit history to find the color), you'll need to do an interactive rebase, check out: https://hackernoon.com/beginners-guide-to-interactive-rebasing-346a3f9c3a6d (I just skimmed this, but it looks pretty good).
Check out our commit discipline page on Read the docs: https://zulip.readthedocs.io/en/latest/contributing/version-control.html#commit-discipline
You can mention in issue with a hash, eg #12532. (You can also do this in your commit message with a keyword to close it, check out: https://help.github.com/en/articles/closing-issues-using-keywords

Try to apply all of the above, and you should be in good shape moving forward.  :)

Thank you so much, @whoodes


Failed to resolve library

hello I am facing this issue while trying to use latest version
failed to resolve: androidx.tonyodev.fetch2:xfetch2:3.2.0

did you resolve this? how i resolve this issue i am stuck from 2 days

use this @SultanAyubi360
implementation("com.github.Akaizz.Fetch:fetch2:3.3")

but how i give my own list to this?

Latest version of Fetch has been released. Please check out the readme and changelog.


Fix: Missing CMS Signature Validation

This PR fixes a missing CMS Signature Validation.

SonarCloud Quality Gate failed.
  0 Bugs
  0 Vulnerabilities
  0 Security Hotspots
  0 Code Smells
 41.7% Coverage
 0.0% Duplication

High Coverage for this PR is hard. I did not found a proper way to manipulate the CMS signature without invalidating the containing certificate.


Issue installing with yarn and npm

Hello, running yarn add @mcxross/zero returned
error An unexpected error occurred: "https://registry.yarnpkg.com/@mcxross%2fzero: Not found".
And npm install @mcxross/zero returned
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@mcxross%2fzero - Not found npm ERR! 404  npm ERR! 404  '@mcxross/zero@*' is not in this registry.
(following installation instructions at https://zeroauth.dev/platform/web/quick-start.html#installation).

Hey,
Public snapshots are coming within a week from now.
But you can still use the package via a Node.js private proxy registry. You can use Verdaccio for this.
Here's how you can do it:
Install Verdaccio:
npm install -g verdaccio

Run it:
$ verdaccio

Set it as a proxy:
npm set registry http://localhost:4873

Build ZeroAuth:
gradlew :zero:jsBrowserDevelopmentWebpack

Publish it:
npm publish

You can then use the package as explained in the docs here

Ah cool, thanks so much.
On Tue, Nov 21, 2023 at 8:11 AM @a @.***> wrote:

Hey,
Public snapshots are coming within a week from now.
But you can still use the package via a Node.js private proxy registry.
You can use Verdaccio for this.
Here's how you can do it:
Install Verdaccio:
npm install -g verdaccio
Run it:
$ verdaccio
Set it as a proxy:
npm set registry http://localhost:4873
Build ZeroAuth:
gradlew :zero:jsBrowserDevelopmentWebpack
Publish it:
npm publish
You can then use the package as explained in the docs here
https://zeroauth.dev/platform/web/quick-start.html#installation
—
Reply to this email directly, view it on GitHub
https://github.com/mcxross/zeroauth/issues/1#issuecomment-1820898590,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEGI267H6ZH7XIVMUBYRYF3YFSR6LAVCNFSM6AAAAAA7TTZCFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRQHA4TQNJZGA
.
You are receiving this because you authored the thread.Message ID:
@.***>


Feature: Allow actions for multiple objects

I am using the pagarme ruby lib to make requests, and sometimes I need to check some informations about my objects on pagarme API.
The problem is that the API does not allow me to find multiple objects by id. Then I need to iterate all my objects, and in each iteration I need to make a request.
It could handle if there was the PUT, POST and GET options for multiple objects.

Hey @Maychell! We just create documentation templates, and aren't associated with the pagarme API. You should contact the people who run that lib/website directly.


kubie and kubens mixed cause kubeconfig syntax error

Hello Guys,
I use your stuff all the time, it ia great stuff. Thanks for it.
I have a little bug. Although it is a little, it is so irritating.
Im using kubie with kubectx and kubens (at the most of time to use kube for set context)
When I start a new terminal with 'kubie ctx' and select a cluster, the proper kube config will be generatred into the /tmp/kubie-.yaml
At this time if I use kubens and select a namespace, the kube config namespace field will be:
❯ kgp
error: error loading config file "/tmp/kubie-configRcB26D.yaml": yaml: cannot decode !!str `kube-node-lease` as a !!null

❯ cat /tmp/kubie-configRcB26D.yaml | grep namespace
        namespace: !!null kube-node-lease

There is a !!null in the namespace field.
After you delete this 6chars ('!!null ') it will be proper and from this time it is working properly. Both kubie, kubens.
For native kubernetes clusters this problem is not showed. The problem is showed only for OpenShift cluters. (It is magic :))
More intersting facts:
kubens cannot do this error without kubie
kubie cannot do this error without kubens
I would like to ask you how can I localize the source of the error?
Do you have any idea what cause this error?
Is it possible to use 'default' instead of '~' character at namespace value field?
Thanks in advance.
BR,
Viktor

go binary

I am quite confused as I havent seen this format before. If you can paste some kubeconfig snippets that can repro the problem, I am happy to take a look.

My main kubeconfig (default location ~/.kube/config) contains 7 clusters. I cant share the datas, but simple multicluster setup, with openshift clusters and vanilla kubernetes.
The problem is showing only first time when move into kubie context. I need to patch file manually and after it everything is fine.
In my opinion, this problematic part could be an 'if', 'loop', 'func' at start, where some checking is processing, and one of it has a null return. it should be near there where you try to make the current context. it is just based on logic, i havent checked the code.
Can you give me advise how can make kubie more verbose logging?
Thanks,
Viktor

it cannot be reproduced without kubens and vica versa.
when kubie starts and make the "new terminal" with different kubeconfig. It is proper.
Is it possible that kubens cannot grant the whole structure when get ~(default) namespace char. Im going to ask them.

Hello,
@ahmetb
I found the exact location of the error:
Kubie generates kubeconfig file with this
namespace: ~

Your kubens code is not tolerating this, it will be fullfilled with this (variable comes from choosen namespace, in this example default):
namespace: !!null default

The problem could be here or here
Would you help me about it?
Thanks,
Viktor

As far as I know neither of those values are known to kubectl either. Not sure why I should be fixing this. Can elaborate what they mean?

I've just suggest to "fix" this, cause honestly it is not a bug in your application. If kubens can handle this kind of behaviour, your app could be more compatible with other tools, maybe it will be more popular (if it counts).
If you are not sure about fixing, I can check it also, and if I'll find something solution (modify code to tolerate '~' this character at namespace) I can contact you, and you can decide to merge or not.

Are you sure kubectl tolerates ~ ?

Im not sure about it, but, Ive tried with my kubeconfig, and it works. It could be a fallback, I dont know cause Google hasnt got any relevant information about this. kubectl config use-context that-one-has-symbol-at-namespace-field is working for me with '~'

Also I see kubie is an alternative to this tool. Not sure why you're using both at the same time. :)


[QUESTION] Export from large array

PHP version 7.1.18.
Laravel version: 5.5.* which previously was 5.4.36.
Maatwebsite\Laravel-Excel 3.1.0.

How can I export large array say with size more than 60,000 into excel?
I don't want to use eloquent as I'm not fetching data from database. I'm following this link.
Mentioned that Query automatically chunks the results for better performance
Here is my export class
<?php
namespace App\Exports;

use Maatwebsite\Excel\Concerns\FromCollection;
use Maatwebsite\Excel\Concerns\Exportable;
use Maatwebsite\Excel\Concerns\WithHeadings;
use Maatwebsite\Excel\Concerns\WithBatchInserts;
use Maatwebsite\Excel\Concerns\WithChunkReading;
use Maatwebsite\Excel\Concerns\FromArray;

class ExportFile implements FromArray, WithHeadings, WithBatchInserts, WithChunkReading
{
    use Exportable;
    private $myArray;
    private $myHeadings;

    public function __construct($myArray, $myHeadings){
        $this->myArray = $myArray;
        $this->myHeadings = $myHeadings;
    }

    public function array(): array{
        return $this->myArray;
    }

    public function headings(): array{
        return $this->myHeadings;
    }
}
?>

Here is my sample controller
<?php
namespace App\Http\Controllers;

use App\Exports\ExportFile;

class MyController extends Controller
{
    public function index(){
        $data = []; //array with size 65345
        $header = []; //headers
        $excel = Excel::download(new ExportFile($data, $header), "excel.xlsx");
        return $excel;
    }
}
?>

My question is does this behaviour remains same, if I use FromArray concerns? How can I use chunking in this manner? Right now it is taking approx. 10 minutes to download excel with 50,000 rows.

PHP version 7.1.18.
Laravel version: 5.5.* which previously was 5.4.36.
Maatwebsite\Laravel-Excel 3.1.0.

How can I export large array say with size more than 60,000 into excel?
I don't want to use eloquent as I'm not fetching data from database. I'm following this link.
Mentioned that Query automatically chunks the results for better performance
Here is my export class
<?php
namespace App\Exports;

use Maatwebsite\Excel\Concerns\FromCollection;
use Maatwebsite\Excel\Concerns\Exportable;
use Maatwebsite\Excel\Concerns\WithHeadings;
use Maatwebsite\Excel\Concerns\WithBatchInserts;
use Maatwebsite\Excel\Concerns\WithChunkReading;
use Maatwebsite\Excel\Concerns\FromArray;

class ExportFile implements FromArray, WithHeadings, WithBatchInserts, WithChunkReading
{
    use Exportable;
    private $myArray;
    private $myHeadings;

    public function __construct($myArray, $myHeadings){
        $this->myArray = $myArray;
        $this->myHeadings = $myHeadings;
    }

    public function array(): array{
        return $this->myArray;
    }

    public function headings(): array{
        return $this->myHeadings;
    }
}
?>

Here is my sample controller
<?php
namespace App\Http\Controllers;

use App\Exports\ExportFile;

class MyController extends Controller
{
    public function index(){
        $data = []; //array with size 65345
        $header = []; //headers
        $excel = Excel::download(new ExportFile($data, $header), "excel.xlsx");
        return $excel;
    }
}
?>

My question is does this behaviour remains same, if I use FromArray concerns? How can I use chunking in this manner? Right now it is taking approx. 10 minutes to download excel with 50,000 rows.

If I'm correct, the concerns for chunking are only applicable to Imports (see the imports concerns list).
The export concerns list doesn't mention anything about chunking.

By using the FromQuery concern, we can prepare a query for an export. Behind the scenes, this query is executed in chunks.

FromArray does not do chunking.

@GlennM thanks, so is there any other way available to archive this?

@GlennM thanks, so is there any other way available to archive this?
As far as I know, unfortunately not. Only FromQuery seems to apply chunking on imports (see Maatwebsite\Excel\Sheet, line 358).

Well thanks a lot @GlennM. It will be better if you take this as a feature request 😃   Thank you.

@abhijeet9920 there's not really a point in chunking FromArray/FromCollection as the results are already held in memory. Chunking those arrays would actually cause a performance decrease as it would have to call PhpSpreadsheet multiple times and still keep the original array in memory.
The reason we do it for queries is because it isn't performant to keep thousands of Eloquent entities in memory. If you choose to use FromArray than there isn't a lot we can do about performance.


Robotics.mobile.core

How to add the reference of robotics.mobile.core. I didnt find this in nuget.

Try Monkey.Robotics


Development

Hammertime development branch

Didnt even know there was a fork of sledge, hope this project continues to be expanded maybe even with hammer prefab support, cause this is looking awesome!

I would if you want ofc, make an improved readme for you, so it doesnt seem like you stopped working on it

Did you add prefab support? Oh my god this is getting better and better, hope to see more cause im already cooking some stuff up

Yes.
You can add prefabs from .ol file, and save them to .ol
But it has only support for 1.6, 1.8 and 2.2 (see sledge-formats/Sledge.Formats.Map/Formats/WorldcraftRmfFormat.cs)
I plan to move prefabs into separate tool, and add support for older versions, for example vhe "old stuff.ol" library wont work, it has 0.9 version of rmf, and i didn't included it into package

Will you add some more features to the orefabs soon? Or is that not a priority

As i've written above - i think it would be better to use prefabs as separate tool.
Need to implement tool for placing prefab into coordinates, not just "spawn" them.
Need to add tool for creating new libraries.
Nothing is in priority, will do something if i could/want.

Since last update my hammertime wont start, it recognizes it as a virus also

Sorry for that, i will try to fix that in with next after next update. I found that today after downloading from github release.
It wont recognize as virus if i build it by myself (not by github actions), so the only advice is to disable windows security realtime scan for  the moment you download, and then just add to exclude list. Or build by yourself.
I'll try to fix that in near future

Alright will try to

Seems like today's update of window security (1.403.3078.0) has fixed the issue with false-positive antivirus reaction
Downloaded by myself, tested on friends pc.

Alright

Also can you look at the pr I made please?

This is really Sledge continue? 😮

Sure, fixing bugs and enhancing when i have free time.
I wont implement new features for week, maybe two, I'm busy rn, will only fix minor bugs

Completely okay take your time

Seems like installation file is working fine, when i get a few hours - would setup an auto build installer && possible fix auto-update

Added installer to Assets, for me - it doesn't trigger Windows Security
But still left both, zip and installer

Since i have renamed AssemblyName - application settings changed its store directory, to restore your settings please run this code:
robocopy %appdata%/sledge/shell/ %appdata%/hammertime/shell/ /it /is /im del %appdata%\hammertime\shell\Hammertime.Shell.json del %appdata%\hammertime\shell\Hammertime.Editor.UpdateChecker.json ren %appdata%\hammertime\shell\Sledge.Shell.json Hammertime.Shell.json ren %appdata%\hammertime\shell\Sledge.Editor.UpdateChecker.json Hammertime.Editor.UpdateChecker.json

Will there be an Option to do some realtime lighting? I want to use this with Godot using a plugin, but I would need realtime lighting previews to actually make it viable, so I dont have to resort to using Hammer++

Maybe.
First of all I need to understand how this works, so, it wont be quick implementation

Will there be an Option to do some realtime lighting? I want to use this with Godot using a plugin, but I would need realtime lighting previews to actually make it viable, so I dont have to resort to using Hammer++

Please open Feature Request issue if you want some new feature


Kernel parameter doesn't exist

The kernel parameter doesn't exist in the container ;)

Agree, this is a service.

fixed :)

thanks @oktapodia I'll hope you will become certified, but not too much because you won't be able to contribute then :cat:

Thank you ;)


Status of the progress and results?

@ThibeautFormesyn @MaartenBlomme @joliedegeyter any progress on the data analysis so far? Any questions or issues that @JanBaert or myself can address?

@stijnvanhoey we are currently testing some different data analysis approaches. At this moment, I don't believe they are specific questions. When we have finished some things, we will show them to you.


Zip operator behavior is not consistent with System.Reactive.Linq

Zip operator is enumerating n-1 observable till completion, before it produces any results.
For example:
using R3;

IEnumerable<int> Series(int mul)
{
    for (int i = 1; i < 10; i++)
    {
        Console.WriteLine("Generating for {0}", mul);
        yield return i * mul;
    }
}

var d = new Observable<int>[] { Series(1).ToObservable(), Series(10).ToObservable(), Series(100).ToObservable(), };
var waitHandle = new ManualResetEventSlim(false);

using var _ = Observable.Zip(d)
    .SubscribeOnThreadPool()
    .Subscribe(x =>
{
    Console.WriteLine("{0}, tid:{1}", string.Join(',', x), Environment.CurrentManagedThreadId);
}, (res) => waitHandle.Set());

waitHandle.Wait();

is producing
Generating for 1
Generating for 1
Generating for 1
Generating for 1
Generating for 1
Generating for 1
Generating for 1
Generating for 1
Generating for 1
Generating for 10
Generating for 10
Generating for 10
Generating for 10
Generating for 10
Generating for 10
Generating for 10
Generating for 10
Generating for 10
Generating for 100
1,10,100, tid:10
Generating for 100
2,20,200, tid:10
Generating for 100
3,30,300, tid:10
Generating for 100
4,40,400, tid:10
Generating for 100
5,50,500, tid:10
Generating for 100
6,60,600, tid:10
Generating for 100
7,70,700, tid:10
Generating for 100
8,80,800, tid:10
Generating for 100
9,90,900, tid:10

However using System.Reactive.Linq doesn't enumerate all the enumerable till completion.
Output using System.Reactive.Linq:
Generating for 1
Generating for 10
Generating for 100
1,10,100, tid:1
Generating for 1
Generating for 10
Generating for 100
2,20,200, tid:1
Generating for 1
Generating for 10
Generating for 100
3,30,300, tid:1
Generating for 1
Generating for 10
Generating for 100
4,40,400, tid:1
Generating for 1
Generating for 10
Generating for 100
5,50,500, tid:1
Generating for 1
Generating for 10
Generating for 100
6,60,600, tid:1
Generating for 1
Generating for 10
Generating for 100
7,70,700, tid:1
Generating for 1
Generating for 10
Generating for 100
8,80,800, tid:1
Generating for 1
Generating for 10
Generating for 100
9,90,900, tid:1

Is there any way to get similar behavior of System.Reactive.Linq?

About Zip, already reported at #162
The current behavior of Zip is not good, and I don't think the one in dotnet/reactive is ideal either.
Regarding this, I definitely want to implement improvements next.

Thanks for your reply. I think problem is much more severe than onComplete not been called. When Zip has infinite streams, it doesn't even produce any subscription callbacks.
For example subscription is not being called in this code:
using System;
#if USER3
using R3;
#else
using System.Reactive.Linq;
#endif

IEnumerable<int> Series(int mul)
{
    for (int i = 1; ; i++)
    {
        Console.WriteLine("Generating for {0}", mul);
        yield return i + mul;
    }
}

var d = new
#if USER3
    Observable<int>[]
#else
    IObservable<int>[]
#endif
{ Series(1).ToObservable(), Series(10).ToObservable(), Series(100).ToObservable(), };

var waitHandle = new ManualResetEventSlim(false);

#if USER3
Console.WriteLine("using R3");
using var _ = Observable.Zip(d)
.SubscribeOnThreadPool()
    .Subscribe(x =>
    {
        Console.WriteLine("{0}, tid:{1}", string.Join(',', x), Environment.CurrentManagedThreadId);
    }, (res) => waitHandle.Set());
#else
Console.WriteLine("using System.Reactive.Linq");
await Task.Run(() =>
{
    Observable.Zip(d)
        .Subscribe(onNext: x =>
        {
            Console.WriteLine("{0}, tid:{1}", string.Join(',', x), Environment.CurrentManagedThreadId);
        }, onError: (err) => { }, onCompleted: waitHandle.Set, CancellationToken.None);
});
#endif
waitHandle.Wait();

I think problem is much more severe than onComplete not been called. When Zip has infinite streams, it doesn't even produce any subscription callbacks.

It is not promblem of Zip, This is due to differences in ToObservable's default behavior.
In R3, the default is the equivalent of Scheduler.Immediate, so changing dotnet/reactive to the following code, for example, will result in similar behavior.
{ Series(1).ToObservable( Scheduler.Immediate), Series(10).ToObservable(Scheduler.Immediate), Series(100).ToObservable(Scheduler.Immediate), };

CurrentThreadScheduler is applied by default in dotnet/reactive, but there is no CurrentThreadScheduler equivalent in R3.

Thank you for the clarification. It would be highly beneficial to have a mechanism to create cold observables from enumerables


Configure SAP API Management policies to avoid CORS issues and generate an API Key

Tutorials: https://developers.sap.com/tutorials/btp-integration-suite-integral-api-management.html
On write "preflight" step, there is a bug on API Management UI, the Update stay on grey/disable , to avoid it and save your changes, you must create a dummy assign message anywhere, save it and remove it.

@mariajosesap can you please respond to this user's feedback?

Hi @bdbais @jmmargo , that's right, there's a bug in the service that prevents from saving/updating the policy when adding the "preflight" condition. The recommendation on how to solve it, it's included in the tutorial. The other option is to create the preflight condition first and then add the setCORS condition in the PostFlow.

Thank you for the solution @mariajosesap.  As such I will close this issue.  @bdbais if you need to reopen this issue, please feel free to do so.


[BUG]: Test Purposes

Bug Description
N/A
Logs
N/A
Screenshots
No response
Additional Environment Information
No response
Steps to Reproduce
N/A
Expected Behavior
N/A
CSM Driver(s)
N/A
Installation Type
No response
Container Storage Modules Enabled
No response
Container Orchestrator
N/A
Operating System
N/A

/sync

/sync

/sync

/sync

/sync


Add Preview docs to README

Adds docs for the Preview component to the README.
I had to dig through the issues and demo example to figure out how to get slides to display in the client, so hopefully this will help the next person who tries!
Also updates the broken image link in the example code to a working link: https://picsum.photos/id/237/460/300

Thanks!


Unable to unformat text

Hi, thanks for the great extension! However I'm still experiencing the same issue that was mentioned in #2 about not being able to unformat text unless you select some text next to it.
I am using Chrome Version 56.0.2924.87 on Mac OS X 10.11.6. This also happens in the demo page:

@ashley71 @slegaitis : Did you discover a workaround?


Core architecture

Is there a UML architecture documentation, for the core application?
Otherwise I can take care of it ?

Closing this one since someone else has taken care of it, the UML will be added to the Product Brief.


dolt: 1.43.15 -> 1.44.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/dolthub/dolt/releases.
meta.description for dolt is: Relational database with version control and CLI a-la Git
meta.homepage for dolt is: https://github.com/dolthub/dolt
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.44.0 with grep in /nix/store/7kknxamifp3cw0v2w1j2996lk9qnnvdi-dolt-1.44.0
found 1.44.0 in filename of file in /nix/store/7kknxamifp3cw0v2w1j2996lk9qnnvdi-dolt-1.44.0

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

dolt

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/7kknxamifp3cw0v2w1j2996lk9qnnvdi-dolt-1.44.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A dolt https://github.com/r-ryantm/nixpkgs/archive/18a0cbaae4380439b1caf5e895485e3f88c5bb9a.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/18a0cbaae4380439b1caf5e895485e3f88c5bb9a#dolt

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/7kknxamifp3cw0v2w1j2996lk9qnnvdi-dolt-1.44.0
ls -la /nix/store/7kknxamifp3cw0v2w1j2996lk9qnnvdi-dolt-1.44.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review

x86_64-linux

:white_check_mark: 1 package built:

dolt

Maintainer pings
cc @danbst for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 363598

aarch64-darwin

:white_check_mark: 1 package built:

dolt


Can the project support Vector stores that i can run and store locally? e.g: ChromaDB.

ChromaDB, I couldn't find the Java SDK. What do you think about Milvus? It can be installed locally with Docker and provides a Java SDK. Can it meet your requirements?

I could take a crack at writing and contributing this integration if it would help. I was already looking into this and possibly OpenSearch as well for storing vectors.

I need your contribution. You can fork the repository and then submit a pull request to the dev branch.

Yes. Milvus works. I would like to store the vector data in my local machine.

@HamaWhiteGG When will Milvus be able to support vector databases?

I am making good progress on this but I will probably need a couple of more weeks. I don't have a lot of time to work on it.

@krishnakiran @odysseaspenta @michael19960921
I have supported Milvus, and you can see MilvusExample.
public static void main(String[] args) {
    var filePath = "docs/extras/modules/state_of_the_union.txt";
    var loader = new TextLoader(filePath);
    var documents = loader.load();
    var textSplitter = CharacterTextSplitter.builder().chunkSize(1000).chunkOverlap(0).build();
    var docs = textSplitter.splitDocuments(documents);

    var embeddings = OpenAIEmbeddings.builder().requestTimeout(60).build().init();

    ConnectParam connectParam = ConnectParam.newBuilder()
            .withHost("<IP_ADDRESS>")
            .withPort(19530)
            .build();

    Milvus milvus = Milvus.builder()
            .embeddingFunction(embeddings)
            .connectParam(connectParam)
            .collectionName("LangChainCollection_1")
            .build().init();
    milvus.fromDocuments(docs, embeddings);

    var query = "What did the president say about Ketanji Brown Jackson";
    docs = milvus.similaritySearch(query);

    var result = docs.get(0).getPageContent();
    println(result);
}

See the following documentation for how to run a Milvus install_standalone-docker

@HamaWhiteGG thanks

@HamaWhiteGG Thanks


present? returns true on some hidden elements

I just upgraded from Watir 6.11.0 to 6.14.0 and replaced visible? occurrences with present?. I got some unexpected behavior with hidden elements (present in DOM but not displayed on screen).
Meta -
Watir Version: 6.14.0
Selenium Version:  ?
Browser Version: Chrome 69.0.3497.100 (Official Build) (64-bit)
Browser Driver Version: ChromeDriver 70.0.3538.16
OS Version: Debian 9.4.0
Expected Behavior -
When an element is not displayed on the page, present? should return false and wait_until_present should time out.
Actual Behavior -
On some hidden elements, present? always returns true although wait_until_present or wait_until(&:present?) do time out.
Steps to reproduce -
Case 1 :
For <div id="some-id" style="display:block;" class="some-class"> then div.present? is true
Case 2 :
For a ul with a class forcing opacity to 0 then ul.present? is true
In both cases, the behavior is different from wait_until_present (timed out after x seconds, waiting for element)

These worked with wait until visible 6.11 and don't work with wait until present 6.14?

Did you use activesupport gem? It has implemented present? method on Object class.
This may make your call to present? method always return true in some cases.

@tobytang1987 you are right, the present? method from activesupport is called instead (with div.present? but not div.wait_until_present). As I was only using visible? method before updating Watir I didn't see this.
Do you know how I make sure the Watir present? is called ?

My bad, the Watir present? method is called correctly.
It turns out I'm using cheezy page-object gem and the issue is in this gem :
https://github.com/cheezy/page-object/issues/465
Thanks for your help.

Ho you were faster than me ;D


Fixed gometalinter:aligncheck

Reduce struct size
Checked with gometalinter --disable-all --enable=aligncheck "./..."

Reviewed, tested, merging. Thank you!


"ImportError: cannot import name" after installing simsopt with conda

I tried to install simsopt on my laptop with conda:
conda install -c hiddensymmetries simsopt
In python I can now import simsopt without any error messages and obtain the version for example:

import simsopt; print(simsopt.version)
0.7.4

But I can't load anything from subpackages:

from simsopt.objectives import SquaredFlux
Traceback (most recent call last):
File "", line 1, in
ImportError: cannot import name 'SquaredFlux' from 'simsopt.objectives'

Is this an installation issue? How can I fix this?

Which machine are you on? If it's a laptop or desktop, which OS?
Can you also provide the output of conda list?
Thanks.

This may be because the imports have been modified since v0.7.4. Try from simsopt.objectives.fluxobjective import SquaredFlux - does that work?

Hi Matt,
You are right. from simsopt.objectives.fluxobjective import SquaredFlux  worked.
Do you know why conda installed an older version?
I am on my laptop which is a linux machine (Manjaro Linux).
My conda list output is:
Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
absl-py                   1.2.0              pyhd8ed1ab_0    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2022.6.15            ha878542_0    conda-forge
cudatoolkit               11.7.0              hd8887f6_10    conda-forge
cudnn                     <IP_ADDRESS>             hed8a83a_0    conda-forge
deprecated                1.2.13             pyh6c4a22f_0    conda-forge
etils                     0.7.1              pyhd8ed1ab_0    conda-forge
f90nml                    1.4.3              pyhd8ed1ab_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gmpy2                     2.1.2            py39h78fa15d_0    conda-forge
grpc-cpp                  1.46.4               h30feacc_6    conda-forge
importlib_resources       5.9.0              pyhd8ed1ab_0    conda-forge
jax                       0.3.17             pyhd8ed1ab_0    conda-forge
jaxlib                    0.3.15          cuda112py39hf9e344c_203    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libabseil                 20220623.0      cxx17_h48a1fff_1    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_2    conda-forge
libprotobuf               3.21.5               h6239696_0    conda-forge
libsqlite                 3.39.2               h753d276_1    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.12               h166bdaf_2    conda-forge
monty                     2022.4.26          pyhd8ed1ab_0    conda-forge
mpc                       1.2.1                h9f54685_0    conda-forge
mpfr                      4.1.0                h9202a9a_1    conda-forge
mpmath                    1.2.1              pyhd8ed1ab_0    conda-forge
nccl                      <IP_ADDRESS>             h0800d71_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nptyping                  2.3.1              pyhd8ed1ab_0    conda-forge
numpy                     1.21.6           py39h18676bf_0    conda-forge
openssl                   3.0.5                h166bdaf_1    conda-forge
opt_einsum                3.3.0              pyhd8ed1ab_1    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
pyevtk                    1.5.0              pyh8a188c0_0    conda-forge
python                    3.9.13          h2660328_0_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
randomgen                 1.23.1           py39hd257fcd_0    conda-forge
re2                       2022.06.01           h27087fc_0    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
ruamel.yaml               0.17.21          py39hb9d737c_1    conda-forge
ruamel.yaml.clib          0.2.6            py39hb9d737c_1    conda-forge
scipy                     1.9.1            py39h8ba3f38_0    conda-forge
setuptools                65.3.0             pyhd8ed1ab_1    conda-forge
simsopt                   0.7.4            py39h6cd0baa_0    hiddensymmetries
sqlite                    3.39.2               h4ff8645_1    conda-forge
sympy                     1.10.1           py39hf3d152e_1    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
typing_extensions         4.3.0              pyha770c72_0    conda-forge
typish                    1.9.3              pyhd8ed1ab_0    conda-forge
tzdata                    2022c                h191b570_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
wrapt                     1.14.1           py39hb9d737c_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zipp                      3.8.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.12               h166bdaf_2    conda-forge
(I now managed to install the new version from the source code using pip install by just following (more or less) the wiki for the "Greifswald cluster". )
Thanks for your help.

Hi @SHenneberg , @mbkumar and I looked at this today, and part of the issue was that the conda package CI wasn't working during the last few releases, hence the version of the conda package was quite old. @mbkumar just created a new release, so now if you try again to install the conda package, the version should be v0.9.3, which should be current. Then from simsopt.objectives import SquaredFlux should work.


BUG: DefaultKafkaProducerFactory creates producers which are never closed

I am using spring-kafka version: 2.6.3 (spring-boot-starter-parent:2.4.0) but the problem should be present in the current version as well.
Preconditions:

The Kafka broker is not available.
spring.kafka.producer.transaction-id-prefix is set in the properties
A Kafka transaction is started, either with @Transactional or with KafkaTemplate.executeInTransaction()

Problem:
The DefaultKafkaProducerFactory will create a new KafkaProducer.
protected Producer<K, V> createRawProducer(Map<String, Object> rawConfigs) {
    KafkaProducer<K, V> kafkaProducer =
            new KafkaProducer<>(rawConfigs, this.keySerializerSupplier.get(), this.valueSerializerSupplier.get());
    this.postProcessors.forEach(pp -> pp.apply(kafkaProducer));
    return kafkaProducer;
}

This KafkaProducer automatically starts a new thread which runs in an infinite while loop until the producer is closed.
Right after, the KafkaProducer.initTransactions() method is called:
private CloseSafeProducer<K, V> doCreateTxProducer(String prefix, String suffix,
		BiPredicate<CloseSafeProducer<K, V>, Duration> remover) {
	Producer<K, V> newProducer;
	Map<String, Object> newProducerConfigs = new HashMap<>(this.configs);
	String txId = prefix + suffix;
	newProducerConfigs.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, txId);
	if (this.clientIdPrefix != null) {
		newProducerConfigs.put(ProducerConfig.CLIENT_ID_CONFIG,
				this.clientIdPrefix + "-" + this.clientIdCounter.incrementAndGet());
	}
	checkBootstrap(newProducerConfigs);
	newProducer = createRawProducer(newProducerConfigs);
	newProducer.initTransactions();
	CloseSafeProducer<K, V> closeSafeProducer =
			new CloseSafeProducer<>(newProducer, remover, prefix, this.physicalCloseTimeout, this.beanName);
	this.listeners.forEach(listener -> listener.producerAdded(closeSafeProducer.clientId, closeSafeProducer));
	return closeSafeProducer;
}

If the Kafka broker is not available, the initTransactions() method will throw a TimeoutException. As a result, this Producer is never closed and the aforementioned thread runs infinitely.
The application does not recover from this even if the Kafka broker comes back, because the DefaultKafkaProducerFactory is not aware of them. This results in "dead" threads running in the background.
Solution:
Make a try-catch for any RuntimeException around newProducer.initTransactions(). In the catch block, close the producer and rethrow the exception.
private CloseSafeProducer<K, V> doCreateTxProducer(String prefix, String suffix,
                                                   BiPredicate<CloseSafeProducer<K, V>, Duration> remover) {
    Producer<K, V> newProducer;
    Map<String, Object> newProducerConfigs = new HashMap<>(this.configs);
    String txId = prefix + suffix;
    newProducerConfigs.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, txId);
    if (this.clientIdPrefix != null) {
        newProducerConfigs.put(ProducerConfig.CLIENT_ID_CONFIG,
                               this.clientIdPrefix + "-" + this.clientIdCounter.incrementAndGet());
    }
    checkBootstrap(newProducerConfigs);
    newProducer = createRawProducer(newProducerConfigs);
    try {
        newProducer.initTransactions();
    } catch (RuntimeException e) {
        newProducer.close();
        throw e;
    }
    CloseSafeProducer<K, V> closeSafeProducer =
            new CloseSafeProducer<>(newProducer, remover, prefix, this.physicalCloseTimeout, this.beanName);
    this.listeners.forEach(listener -> listener.producerAdded(closeSafeProducer.clientId, closeSafeProducer));
    return closeSafeProducer;
}

Thank you for the quick response and fix. This is amazing. Can I expect this fix to be included in 2.7.0-RC1? If so, when does it release? I want to remove my own workaround from our code base asap :)

@NilValue You should check the milestones for scheduled release dates, or the Spring Calendar.
2.7.0-RC1 and 2.6.7 are currently scheduled for March 17.

@garyrussell Okay thank you very much :)


Adds bloom filter aggregator to 'druid-bloom-filters' extension

This PR, building on top of the work introduced in #6222, extends druid-bloom-filters with an aggregator, allowing bloom filters to be computed from query results which can then be used as input to bloom filters in subsequent queries.
example query:
{
  "queryType": "timeseries",
  "dataSource": "wikiticker",
  "intervals": [ "2015-09-12T00:00:00.000/2015-09-13T00:00:00.000" ],
  "granularity": "day",
  "aggregations": [
    {
      "type": "bloom",
      "name": "userBloom",
      "maxNumEntries": 100000,
      "field": {
        "type":"default",
        "dimension":"user",
        "outputType": "STRING"
      }
    }
  ]
}

example results:
[{"timestamp":"2015-09-12T00:00:00.000Z","result":{"userBloom":"BAAAJhAAAA..."}}]

This PR is dependent on #6546 being resolved.


compatibility with a new version of bot_toast

Could you please provide some additional detail about your issue?

Without additional information, I have to close this issue. Feel free to reopen!


Fasta parser shuffle

there are cases where shuffling reads is not desired. An example is parsing fasta files from Bonito where correspondence between reads of different fasta files need to be preserved. This PR adds a flag to allow disabling shuffling in fasta parser. By default, this flag is set to true, i.e. won't impact previous codes in anyway.

Great! I was thinking about implementing this as well. Shuffling can also cause problems with our testing


Sdl.Web.Common.Models.PageModel.GetXpmMarkup() calls unused XpmMetadata

Sdl.Web.Common.Models.PageModel.GetXpmMarkup() tries to retrieve the "CmsUrl" from the XpmMetadata. When this fails it will use the core.cmsurl config value from the Localization
        if (XpmMetadata.TryGetValue("CmsUrl", out cmsUrlValue))
        {
            cmsUrl = (string) cmsUrlValue;
        }
        else
        {
            cmsUrl = localization.GetConfigValue("core.cmsurl");
        }

The CmsUrl in the XpmMetadata is never used. We get the CMS URL from the web.config or from the Topology Manager.
Also, I think it is not logical to try to get the system wide CMS URL from the XpmMetadata. The latter has the page scope and we cannot have a different CMS URL for each page.
So I think we can safely remove the if (XpmMetadata.TryGetValue("CmsUrl", out cmsUrlValue)).

Created internal issue: https://jira.sdl.com/browse/TSI-2899

Fixed in DXA 1.8 and 2.0


Added Dimensionality Reduction/PCA/MiniBatchSparsePCA_RobustScaler.ipynb

for explained variance function - Rescaled data has not been passed


Check data for low p-value when N is low

The p-value is low because the Cox regression is done on thousand of individuals, here N is only the number of individuals having the prior and outcome events.


Feature: Save last terminal configuration

On atom close and reopening, could be nice to have the terminal configuration reopen, just as it was previously set up with terminal names, colors.

I was about to ask the same thing!
Maybe using some code from old plugins like remember-window or remember-session would make it easier to implement?

Me too, that's why I'm here! :smile:

+1

+1

+1

+1
This would be very useful for me because I want to name this TERMINAL by default and set it to green

I came for this as well +1

+1

That would be awesome, it's something that I really need +1


Include [TypeForwardedTo] in a.html

I'm trying to make Ref12 support .Net Core projects.
However, many members (eg, System.Span<>) are defined in System.Runtime the reference assembly, but is forwarded (via [TypeForwardedTo]) to System.Private.CoreLib in the actual source.
Ref12 only sees it as being in System.Runtime, so it opens https://source.dot.net/System.Runtime/Ad.html#d2517139cac388e8, which does nothing.
Can you please change the indexer to process [TypeForwardedTo] attributes and generate links to either the filename or the corresponding a.html in the target project?

@KirillOsenkov Any luck with resolving forwarded types? Ref12 is one of the most useful and frequent extensions I use.

@SLaks @Gav-Brown can one of you folks provide a very detailed repro of what's not working as expected? Which codebase, which symbol, which hyperlink? It'll help me investigate. Thanks!

@KirillOsenkov, @SLaks can probably describe it better than I can, but the following code illustrates what I see.

Install the Ref12 Visual Studio extension.
Create a new solution with each dll type below.
Paste in the same code to each project.

using System.IO;

namespace SomeDll
{
    public class Class1
    {
        public static void TestRef12()
        {
            string a = Path.Combine("hi", "there");
        }
    }
}

Right click on Path.Combine
Select 'Goto Definition'

.NET Framework dll
Expected: Opens ReferenceSource website on 'Combine' method (.NET Framework version).
Actual: Works as expected and opens
https://referencesource.microsoft.com/#mscorlib/system/io/path.cs,2d7263f86a526264
.NET Core dll
Expected: Opens ReferenceSource website on 'Combine' method (.NET Core version).
Actual: Opens Metadata
#region Assembly System.Runtime.Extensions, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.netcore.app\2.2.0\ref\netcoreapp2.2\System.Runtime.Extensions.dll
#endregion

.NET Standard dll
Expected: Opens ReferenceSource website on 'Combine' method.
Actual: Opens Metadata
#region Assembly netstandard, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
// C:\Program Files\dotnet\sdk\NuGetFallbackFolder\netstandard.library\2.0.3\build\netstandard2.0\ref\netstandard.dll
#endregion

In this case there is no "source" but perhaps Ref12 could be enhanced to select a default implementation to show? .NET Framework or .NET Core source? I would certainly find this more useful than just the metadata.
p.s. I'm not sure if ReferenceSource website has .NET Core source, so perhaps GitHub instead?
Hope this helps.

Correction. .NET Core source should be via source.dot.net

As Microsoft are retiring .NET Framework in favour of .NET Core (.NET 5), for .NET Standard projects I would just stick to the .NET Core source. No need to update Ref12 to specify choice.

Apologies I never followed up.
I'm not sure how to get from System.Runtime.Extensions.dll reference assembly to System.Private.CoreLib. There isn't a TypeForwardedTo attribute that would bridge the two.
I'll ask around and see what can be done here.

@SLaks As an interim solution, for .NET Standard and .NET Core projects, could Ref12 be altered to open source.dot.net with the search parameter set to Path.Combine e.g. https://source.dot.net/#q=Path.Combine
This isn't as streamlined as opening the source page directly, but adds more value than the metadata.


Add /silent, /verbose and /ForceReload

The commits do the following:

Add /silent which disables the MsgBoxes and instead outputs Util_Error to stderr and Util_Info to stdout.
Add /ForceReload which forces the script to reload if it's running, it uses Process Close instead of WinClose.
Add /verbose to output Util_Status to stdout.
Remove Error:  from the error string in all BadParams() calls, since the title of the MsgBox already says Ahk2Exe Error as does the stdout string.
Make the AhkCompile() function return the path and name of the compiled .exe, this fixes the blank ExeFile variable when /out is not used.

I have checked the "Allow edits by maintainers" box, so you can modify the changes however you see fit.

@TAC109 @Lexikos
Thoughts? Most of the changes make sense to me, but I am hesitant with a little few of them since I have not encountered their use or case... either not yet or that I remember, such as the reload message (like #75).

@SaifAqqad
Okay, looks pretty good to me. 👍
I'd like to wait for @TAC109 to take a quick look at it first.

@joedf Now that ConsoleApp is removed I think there might be a problem with outputting errors only to stderr, as you can't see the errors with piping (| more).

@joedf Now that ConsoleApp is removed I think there might be a problem with outputting errors only to stderr, as you can't see the errors with piping (| more).

Yeah, that's the issue... If we don't use Console subsystem then the output is under a "layer" so it quite doesn't "reach" the - console output... I think there's a way around it, but it doesn't come to mind right now...

I think we could either output errors to both stdout and stderr, or we could revert it to how it was before, which is stdout only.

Now that ConsoleApp is removed I think there might be a problem with outputting errors only to stderr, as you can't see the errors with piping (| more).

2>&1 | more will pipe both stderr and stdout to more.
2>fileA >fileB would direct stderr to fileA and stdout to fileB.
Console apps inherit the console of the parent process if it has one, and by default have stdout and stderr set to the console. GUI apps do not have a console or any std handles by default; i.e. GetStdHandle(STD_ERROR_HANDLE) returns NULL unless the parent process specified a stderr handle when it called CreateProcess, or the process called SetStdHandle.
This will output to stderr if it exists, otherwise to stdout:
try
    FileAppend Error to stderr`n, **
catch
    FileAppend Error to stdout`n, *

This will output to stderr if it exists, otherwise to stdout:
try
    FileAppend Error to stderr`n, **
catch
    FileAppend Error to stdout`n, *

I just tested this, it works perfectly with 2>&1 | echo

@TAC109 I'm thinking to merge this is if there are no objections. I think it's in a good state now.
I'd like your 'okay' on this, if you're available. If not, I can do further testing and do a more thorough review... to then finally approve and merge this.

Is there any reason why this has not been reviewed by @TAC109 already?

@SaifAqqad Apologies for the delay, but I have been busy adding changes required for the upcoming v2 beta releases. Also attempting to squash a couple of bugs as well. I hope to review your changes soon.
Cheers

@SaifAqqad Apologies for the delay, but I have been busy adding changes required for the upcoming v2 beta releases. Also attempting to squash a couple of bugs as well. I hope to review your changes soon.
Cheers

No worries then, take your time :D

@SaifAqqad
I've had a preliminary look at this PR and have noted that it includes many changes which are not directly related to the original purpose of this PR:

Many error messages have been changed for some reason. Ahk2Exe has it's own style of error messages which has served well in the past, and this should not be changed now.
For some reason this PR also includes a patch for issue #75. The author of this issue has been given several alternate solutions some of which he has found acceptable, and so there is no need for a patch to be included for this problem. I am against complicating the user interface with little-used options when there are other acceptable alternatives available.
There is a patch to include percent signs around the exitcode variable used in Exit and ExitApp statements. This is unnecessary as these statements accept expressions directly.

After these alterations have been made and your fork updated with the recent changes I have made here, I will review this PR in more detail.
Cheers

@TAC109
I've applied your latest changes, restored all error messages and removed all of the /ForceReload code and other unnecessary changes (like the percent signs).


Create unit test and linting jobs with GitHub Actions

Summary
Fixes #6561
As issue
Occurred changes and/or fixed issues

Added GitHub Actions on push
Removed Drone CI
Collected coverage artifacts

Technical notes summary

Set separated jobs for unit and linter
Allow parallelization (as default in GA)
Corrected coverage path to include also JS and TS files, including all the packages

Areas or cases that should be tested

Jobs should be run also on PR
Check below should be present and be green

Areas which could experience regressions

Screenshot/Video

I was wondering if we should not rephrase the checks in something like:

pull request/e2e
pull request/unit-test
pull request/i18n
pull request/lint

Then we could create composite actions for each task and put in the file, so it would be always separated by instance. As it is now is quite redundant. It would allow us also to avoid the skip one.


Build problem

I am living same issue with that: https://github.com/maccoylton/esp-homekit-air-quality-sensor/issues/9
I cloned it with that code: git clone --recursive https://github.com/maccoylton/esp-homekit-multi-sensor.git .
Then, entered src folder and tried to execute that code: make -C . test
Result:
vagrant@vagrant-ubuntu-trusty-64:~/esp-homekit-demo/examples/esp-homekit-multi-sensor/src$ make -C . test
make: Entering directory `/home/vagrant/esp-homekit-demo/examples/esp-homekit-multi-sensor/src'
/home/vagrant/esp-open-rtos/common.mk:223: /home/vagrant/esp-homekit-demo/examples/esp-homekit-multi-sensor/components/esp-homekit/component.mk: No such file or directory
make: *** No rule to make target `/home/vagrant/esp-homekit-demo/examples/esp-homekit-multi-sensor/components/esp-homekit/component.mk'.  Stop.
make: Leaving directory `/home/vagrant/esp-homekit-demo/examples/esp-homekit-multi-sensor/src'
vagrant@vagrant-ubuntu-trusty-64:~/esp-homekit-demo/examples/esp-homekit-multi-sensor/src$

This is dependant on you  have a working esp-open-rtos build environment, instruction for that can be found here:- https://github.com/SuperHouse/esp-open-rtos.
Alternatively check out the wiki for instructions on installing a pre-built firmware

Alternatively check out the wiki for instructions on installing a pre-built firmware

That is amazing, solved my problem quickly.
Thanks a lot ✋🏻


Issues with SPA

Following on from here: https://github.com/vite-pwa/sveltekit/issues/52
I ran into more issues, specifically that loading routes from the server other than the home screen (which had been precached) causes issues as they have not been precached. It's not possible for me to precache all routes because some are dynamic.
To be honest, I don't know if this is a problem relating to this package or sveltekit, and in the end I've migrated everything to pure svelte, instead of using sveltekit. But I thought I would just notify you here in case it's of interest to you.

@hamishjohnson you should exclude any dynamic route adding offline support for those routes or api calls


Add dnb_investigation_id field ahead of new investigation API endpoint

Description of change
This PR adds a new field to the Company model: dnb_investigation_id.  This field will store a UUID generated by dnb-service (https://github.com/uktrade/dnb-service/blob/master/company/models.py#L351) when dnb-service's new "company investigation" API endpoint is called.
This PR also adds some TODO comments to indicate that certain existing company investigation logic/fields will be removed in future.
Background
Blue team is working on a feature to improve the way that new company investigations are created and managed with D&B.  This feature relates to the situation where a user creates a new company on Data Hub and cannot find the company that they are looking for through D&B's search results.
Currently, a stub company is created and an email notification is triggered to launch an investigation with D&B.  This solution is not scalable and we need to move to a batched process - where investigations are queued up and sent to D&B daily in a CSV spreadsheet.
The new solution involves adding an extra "company investigation" endpoint to dnb-service.  The aspiration is that this endpoint will be called every time we need to launch a new company investigation.  Investigations will be persisted on dnb-service and will be submitted to D&B daily in a batched spreadsheet.
Data Hub's part in this is as follows:

It will provide a proxy company investigation endpoint which will allow Data Hub frontend to create new investigations relating to an existing Data Hub company (this company might have just been created as a stub during the company creation unhappy path).  This endpoint will transform and proxy calls through to D&B in a very similar way to the D&B change-request endpoint.
It will record the company investigation ID provided by dnb-service so that investigation responses from D&B can be matched up to a Data Hub company.  Responses from D&B will be handled manually initially, but eventually the aspiration is to handle responses from D&B in a more automated way.
Once this new approach for creating investigations is in use, we will remove the dnb_investigation_data and notify call which relates to the existing way of creating D&B company investigations.

This PR lays the groundwork for the proxy API - which will follow in a subsequent PR.
Checklist

[x] If this is a releasable change, has a news fragment been added?

Explanation
A news fragment is required for any releasable change (i.e. code that runs in or affects production) so that a corresponding changelog entry is added when releasing.
Check changelog/README.md for instructions.

[x] Has this branch been rebased on top of the current develop branch?

Explanation
The branch should not be stale or have conflicts at the time reviews are requested.

[x] Is the CircleCI build passing?

General points

Other things to check

Make sure fixtures/test_data.yaml is maintained when updating models
Consider the admin site when making changes to models
Use select-/prefetch-related field lists in views and search apps, and update them when fields are added
Make sure the README is updated e.g. when adding new environment variables

See docs/CONTRIBUTING.md for more guidelines.

Rebasing to change field type from CharField to UUIDField.


Flutter driver appears to be broken on web

This is causing tree closures on engine and plugins
/cc @jonahwilliams (because I think it might be related to your nullsafety PR)
@jason-simmons is bisecting right now AFAIK.
/cc @zanderso @liyuqian @yjbanov

If we do not currently have a flutter driver based web test in the framework we should add one, and if we do we need to add one that fails the way the engine/plugins repo is seeing.

Re-opening and reducing priority to p1 per @jonahwilliams' comment in linked PR (we need to get test coverage for this failure in the framework repo)

This is blocking https://github.com/flutter/flutter/pull/67468 too. (I was just about to file an issue 😅 )

I am adding a driver test in https://github.com/flutter/flutter/pull/67324, but I thought we already had one.
/cc @nturgut @angjieli

Assigned @yjbanov for this one. please let me know when it is landed so I can re-land the driver migration

We already have a test for Flutter Driver for web: https://github.com/flutter/flutter/tree/master/examples/hello_world/test_driver. I'm not sure how my additional test (which I'm writing with a totally different motivation) can help.

We reverted the driver migration because it apparently broke flutter web driver tests, though nothing in the framework failed.

So we either need a framework test that exercises the driver in the same way to trigger a failure, or all web driver tests need to be deleted. @dnfield can you share the error logs from the test that failed?

These are the error logs from #67468 https://cirrus-ci.com/task/5921209307103232

Perhaps the problem is that we're not running this test on engine PRs?

Nothing in the framework repo failed on postsubmit for that migration PR

Based on the error log provided by @Piinks, the test is failing because we do not have a ChromeDriver running.
Do we have more logs?

From the logs, it seems we fail to fetch the chromedriver, but then keep on trucking:
lib/web_driver_installer.dart: Warning: Interpreting this as package URI, 'package:web_driver_installer/web_driver_installer.dart'.
INFO: chrome version in use Google Chrome 86.0.4240.75
downloading file from https://chromedriver.storage.googleapis.com/null/chromedriver_linux64.zip
Exception during install Exception: Failed to unzip the downloaded Chrome driver chromedriver/chromedriver_linux64.zip.
With the driver path chromedriver
The unzip process exited with code 9.

At this point the test should have stopped. But it kept going:
chromedriver/chromedriver --port=4444 &
sleep 5
/tmp/scripts5526a41a9504680b4e7c8b763a1b1d49.sh: line 11: chromedriver/chromedriver: No such file or directory
cd ../../../examples/hello_world/
flutter drive --target=test_driver/smoke_web_engine.dart -d web-server --profile --browser-name=chrome
Downloading Web SDK...                                              2.3s

Ran out of resource and couldn't extract the chromedriver?

https://github.com/flutter/flutter/pull/67564 should make such failures clearer (but does not fix this issue)

Ran out of resource and couldn't extract the chromedriver?

Doesn't look like it. It simply tries to download from an invalid URL: https://chromedriver.storage.googleapis.com/null/chromedriver_linux64.zip (note the "null" in the URL)

I know what happened. I think we rebuilt the Docker image and picked up Chrome 86, for which we don't have a chromedriver yet. https://github.com/flutter/web_installers/blob/master/packages/web_drivers/driver_version.yaml

https://github.com/flutter/web_installers/pull/21 should temporarily fix the driver issue. However, this is not sustainable. I will file an issue to improve it.

Filed https://github.com/flutter/flutter/issues/67569

web_smoke_test is passing in https://github.com/flutter/flutter/pull/67468 now. Closing.

Re-opening to track failures related to felt test --browser=firefox.
@jonahwilliams - if you re-land the NNBD changes, we need to make sure they don't break firefox.

@yjbanov can you find an owner for the FF investigation?

The Firefox issue happens when running dev/felt.dart test --browser firefox using a framework tree containing https://github.com/flutter/flutter/commit/d411242468773a800945f0d8c6cfcf40c7ea8e44
Example log:
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8867140806173595232/+/steps/felt_test_firefox/0/stdout
The test starts showing a stream of errors like:
1602033985667	webdriver::server	DEBUG	-> POST /session/784010f5-2515-4406-ab3b-319f3e2547ad/execute/sync {"script":"return $flutterDriverResult","args":[]}
1602033985668	Marionette	DEBUG	0 -> [0,8,"WebDriver:ExecuteScript",{"args":[],"script":"return $flutterDriverResult"}]
1602033985688	Marionette	DEBUG	0 <- [1,8,{"error":"javascript error","message":"ReferenceError: $flutterDriverResult is not defined","stacktrace":"@http://localhost:46303/:2:7\n@http://localhost:46303/:3:8\n"},null]
1602033985689	webdriver::server	DEBUG	<- 500 Internal Server Error {"value":{"error":"javascript error","message":"ReferenceError: $flutterDriverResult is not defined","stacktrace":"@http://localhost:46303/:2:7\n@http://localhost:46303/:3:8\n"}}
JavaScript warning: http://localhost:46303/main.dart.js, line 3: unreachable code after return statement

Comparing a good vs a bad execution, it seems they start to differ here:
Bad:
1602034405337	Marionette	DEBUG	0 -> [0,7,"WebDriver:ExecuteScript",{"args":[],"script":"window.$flutterDriver('{\"command\":\"request_data\",\"timeout\":\"60000\",\"message\":\"null\"}')"}]
JavaScript error: http://localhost:39851/main.dart.js, line 5148: : Bad state: This type of status does not exist: null
1602034405354	Marionette	DEBUG	0 <- [1,7,null,{"value":null}]

Good:
1602172895837	Marionette	DEBUG	0 -> [0,7,"WebDriver:ExecuteScript",{"args":[],"script":"window.$flutterDriver('{\"command\":\"request_data\",\"timeout\":\"60000\",\"message\":null}')"}]
1602172895850	Marionette	DEBUG	0 <- [1,7,null,{"value":null}]

Investigating:
JavaScript error: http://localhost:39851/main.dart.js, line 5148: : Bad state: This type of status does not exist: null

I think the NNBD issue

I am adding a driver test in #67324, but I thought we already had one.
/cc @nturgut @angjieli

We had a driver tests. The issue is the way we are using flutter driver with e2e. Since it is unique to engine integration tests we haven't had any test on the flutter/flutter repo. It was obviously a mistake though. I'll copy one of the engine integration tests to the flutter flutter repo to guard against the future changes.

More observations

even though this issue is discussed as firefox, the problem would have occurred in chrome as well. On LUCI recipe we run firefox step earlier than the chrome step therefore the error was on firefox.
all PRs that are synced to head should be good to now.
I'll add a test to flutter/flutter to make sure we can resubmit https://github.com/flutter/flutter/pull/67441 without any breakages.

I'm dropping the priority.

The State Error is being thrown from here:
https://github.com/flutter/plugins/blob/master/packages/integration_test/lib/common.dart#L206
(I'm still not sure how we're getting to the point that we get a null DriverTestMessage)

OK, I've added a bunch of string operations (toUpperCase, length, and runtimeType) to the status variable, and it seems we're indeed receiving a 'null' String (and not proper null):
Bad state: This type of status does not exist: NULL (String, 4 characters)

We are registering the web service extension here: https://github.com/flutter/plugins/blob/master/packages/integration_test/lib/_extension_web.dart#L21
Looks like we should reconsider this line if we want a correct javascript added to the browser

Another solution, we can also check for 'null' in the integration_test package? wdyt?

Good find! Looks like the fix would be to avoid inserting the message if it is null, instead of converting to a string.
Is there a test running in the repo currently that exercises this?

(I double-checked the calls above, and the difference between \"null\" and null is now very painfully obvious 🤦♂

Good find! Looks like the fix would be to avoid inserting the message if it is null, instead of converting to a string.
Is there a test running in the repo currently that exercises this?

Nope, I'm adding one but these are the changes I need to make so it might take some time:
-> add a new validation to the recipes
-> add this as a new builder to flutter/infra
-> add this change to try builds under flutter/flutter

Just ran @jonahwilliams' fix, and this is the log now:
   "script": "window.$flutterDriver('{\"command\":\"request_data\",\"timeout\":\"60000\"}')"

No null or \"null\", but accessing a non existing key from a map returns null, so we're good!

I'm adding tests to make sure this won't happen again. I will be referencing this issue. Please feel free to add comments on the PRs!

The tests are up and running https://ci.chromium.org/p/flutter/builders/prod/Linux web_e2e_test/2


feat(pkg/web): Add a test for the crane url to avoid Grafana 404.

What this PR does / why we need it:
Add a test for the crane url to avoid Grafana 404.
Which issue(s) this PR fixes:

/LGTM


anic: runtime error: invalid memory address or nil pointer dereference

ERROR : Error parsing "An%20Introduction%20to%20Computer%20Viruses%20(and%20other%2.pdf" as URL: parse An%20Introduction%20to%20Computer%20Viruses%20(and%20other%2.pdf: invalid URL escape "%2."
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x64ce36]
goroutine 8804 [running]:
net/url.(*URL).ResolveReference(0xc4223fa700, 0x0, 0xe521f2)
/opt/go/go1.8/src/net/url/url.go:944 +0x36
github.com/ncw/rclone/http.urlJoin(0xc4223fa700, 0xc4205efb80, 0x40, 0x40)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:71 +0x15c
github.com/ncw/rclone/http.parseName(0xc4223fa700, 0xc4223d50e0, 0x4e, 0xc4205efaaf, 0x10, 0xc4205ef901)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:203 +0x6c
github.com/ncw/rclone/http.parse.func1(0xc42004a930)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:243 +0x18d
github.com/ncw/rclone/http.parse.func1(0xc42004a8c0)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse.func1(0xc42004a770)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse.func1(0xc4204b3810)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse.func1(0xc4204b37a0)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse.func1(0xc4204b3030)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse.func1(0xc4204b2cb0)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse.func1(0xc4204b2b60)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:252 +0x147
github.com/ncw/rclone/http.parse(0xc4223fa700, 0x7f39f307d7d8, 0xc4204efae0, 0xc4205ef900, 0x4, 0x4, 0x2, 0x2)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:255 +0x109
github.com/ncw/rclone/http.(*Fs).readDir(0xc420173b80, 0xc4202b8000, 0x2e, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:278 +0x40f
github.com/ncw/rclone/http.(*Fs).List(0xc420173b80, 0xc4202b8000, 0x2e, 0x10, 0xcba7c0, 0xcb1580, 0xc420376360, 0xc4205bbed0)
/home/ncw/go/src/github.com/ncw/rclone/http/http.go:301 +0xc3
github.com/ncw/rclone/fs.ListDirSorted(0x13127c0, 0xc420173b80, 0x428b00, 0xc4202b8810, 0x2d, 0xc42030bd10, 0x737b23, 0xc4205bbf48, 0xc4205bbf40, 0xc4202fe480)
/home/ncw/go/src/github.com/ncw/rclone/fs/operations.go:615 +0x63
github.com/ncw/rclone/fs.(*syncCopyMove).makeListDir.func1(0xc4202b8810, 0x2d, 0xc424224a30, 0x0, 0x1301fe0, 0xc420135ce0, 0x732eb6)
/home/ncw/go/src/github.com/ncw/rclone/fs/sync.go:134 +0x53
github.com/ncw/rclone/fs.(*syncCopyMove)._run.func1(0xc424224a30, 0xc4204ef8a0, 0xc4242249f0, 0xc4201bf040, 0xc4202b8810, 0x2d, 0x7ffffffc, 0x0, 0x100)
/home/ncw/go/src/github.com/ncw/rclone/fs/sync.go:923 +0x72
created by github.com/ncw/rclone/fs.(*syncCopyMove)._run
/home/ncw/go/src/github.com/ncw/rclone/fs/sync.go:924 +0x987
When filing an issue, please include the following information if possible as well as a description of the problem.  Make sure you test with the latest beta of rclone.
https://beta.rclone.org/
https://rclone.org/downloads/

If you've just got a question or aren't sure if you've found a bug then please use the rclone forum instead of filing an issue.

What is your rclone version (eg output from rclone -V)

rclone v1.37

Which OS you are using and how many bits (eg Windows 7, 64 bit)

CentOS6.9

Which cloud storage system are you using? (eg Google Drive)
http -> Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)
Rclone copy
A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

Can you try with the latest beta - I think this is fixed already.

Please re-open if it isn't fixed with the latest beta.  Thanks


rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8"}

rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8"}
Call the rpc to return this error
What is the reason for this? The encoding I requested has been converted to utf-8

Encountered the same problem, again request will be normal. No reason was found.


BusinessForm: Representative Prefix options limited to "Mrs."

In the BusinessForm, the options for "Prefix" in the representative portion of the form include "Mrs." only. We should add all prefixes supported by the API to the dropdown.

Closing this because it was actually fixed a while back. The customer who reported this issue has updated to the latest version of the component and is no longer seeing this issue


Update apache Dockerfile

feature: add arm64 support

I added arm64 architecture support. It works on my mac with Apple silicon (M2). But I can't check it on amd64

Great, thanks!


scripts/reset-password still contain mongo db

scripts/reset-password still contain mongo db

Deleted the file as it's an extra file, no use for it.


ZeRO3, improved parameter all-gather operation

For bing_bert model with following configuration (about 5.1B params), forward computation improved from ~470ms to ~270ms, backward time improved from ~900ms to ~670ms. (hardware setup: 1x ec2-p4d.24xlarge instance)
    "bert_model_config": {
        "vocab_size_or_config_json_file": 32003,
        "hidden_size": 2560,
        "num_hidden_layers": 64,
        "num_attention_heads": 40,
        "intermediate_size": 10240,
        "hidden_act": "gelu",
        "hidden_dropout_prob": 0.1,
        "attention_probs_dropout_prob": 0.1,
        "max_position_embeddings": 512,
        "initializer_range": 0.02
    },

Removing the norm computation in debug printing
Changing _all_gather to be sync op in fetch_sub_module
Reason: the async version is not async at all, because each
all_gather calls torch.cuda.synchronize() to guarantee previous
communication op to be completed
Adding new function _allgather_params_coalesced
the existing _allgather_params has explicit memcpy after the
all-gather op. We can avoid the explicit memory copy at
python side, to improve the performance.
changed _partition_param with torch.empty function.

Notes:
Using most recent updates on PyTorch _all_gather_base function could get further performance boost.
As the _all_gather_base function avoids the redundant memory copy. Refer to https://github.com/pytorch/pytorch/pull/56315

@jfc4050 @tjruwase
I have pushed the customized all_gather operation.
The op uses the cuda stream specified by torch.cuda.stream, and it returns a handle with wrapped cuda event. so you can query/synchronize/wait of the communication on the given stream.
The performance of micro-benchmark shows about 1.1ms time reduction for allgathering a transformer layer with 9.8M params (half-precision) on each partition on a p4d.24xl instance.
In end to end training, the forward time could be further reduced to 200ms for 5.1B bing-bert model. (previously forward time around 280ms-300ms)

Hi,
I would like to remove the customized c++ extension for all-gather operation, due to the hanging issue I observed using customized all-gather wrapper. and left the python part.
The reason for hanging: the customized all-gather is running on a separate cuda-stream, compared to torch.distributed. APIs. Thus, it is easy to have two nccl-communicators working, which cause potential hanging.

@zarzen, thanks for the update. Do you have an idea of the perf impact? Have you pushed the changes into this PR? Also, is this PR ready for review or are you still testing?

@zarzen, another thought is whether the in-place allgather should be PR'd into pytorch?

@tjruwase for the performance impact, the end-to-end training probably drops ~80-100ms in forward and backward, in my setup. I haven't benchmarked yet with all other optimizations I can do. I was working on fixing the hanging issue.
The hanging issue only shows up when I moved the prefetching function to a separate Python thread, so I thought the hanging might due to the size mismatching of collective operations.
I haven't pushed the change yet. I will do the change probably by the end of this week.
I agree the in place all-gather implementation is better to go Pytorch.
Thus, I submitted an issue here https://github.com/pytorch/pytorch/issues/61781 and drafted an implementation.
The master branch of PyTorch has an in-place all-gather, called _all_gather_base, while it only supports a single tensor as output. That's why I am suggesting having a list of tensors as outputs tensor for the _all_gather_base function.

Hi, I have removed the c++ extension, this is ready for review.

Their are 15 failed unit tests
FAILED tests/unit/test_checkpointing.py::test_checkpoint_unfused_optimizer
FAILED tests/unit/test_config.py::test_batch_config[2-32-16-1-True]
FAILED tests/unit/test_config.py::test_batch_config[2-32-18-1-False]
FAILED tests/unit/test_configurable_parallel.py::TestConfigurableMP::test_gpt2_basic
FAILED tests/unit/test_configurable_parallel.py::TestConfigurableMP::test_gpt2_mp2_no_resize
FAILED tests/unit/test_configurable_parallel.py::TestConfigurableMP::test_gpt2_mp_2to1
FAILED tests/unit/test_configurable_parallel.py::TestConfigurableMP::test_gpt2_mp_2to4
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_pp_basic
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_gpt2_mp1_pp_2to1
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_gpt2_mp1_pp_2to4
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_gpt2_mp2_pp_2to1
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_gpt2_mp2_pp_1to2
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_gpt2_pp_2to1_mp_2to1
FAILED tests/unit/test_configurable_parallel.py::TestConfigurablePP::test_gpt2_pp_1to2_mp_1to2
FAILED tests/unit/test_topology.py::test_stage_to_global

Failures in test_configurable_parallel.py all due to megatron import error.
Seems not relevant to this PR

Yes, you can ignore those unit tests. We need to skip them.

I have did perf test, but on V100 GPU rather than A100s, due to the aws capacity issue.
I can still see about ~300ms reduction in forward pass and ~250ms reduction in backward.

What model size?

What model size?
~5.1B params

    "train_batch_size": 512,
    "train_micro_batch_size_per_gpu": 8,
    "steps_per_print": 100,
    "prescale_gradients": false,
    "bert_token_file": "bert-base-uncased",
    "bert_model_config": {
        "vocab_size_or_config_json_file": 3.200300e+04,
        "hidden_size": 2.560000e+03,
        "num_hidden_layers": 64,
        "num_attention_heads": 40,
        "intermediate_size": 1.024000e+04,
        "hidden_act": "gelu",
        "hidden_dropout_prob": 0.1,
        "attention_probs_dropout_prob": 0.1,
        "max_position_embeddings": 512,
        "initializer_range": 0.02
    },
    "zero_optimization": {
        "stage": 3,
        "overlap_comm": true,
        "reduce_scatter": true,
        "contiguous_gradients": true
    },
    "zero_allow_untested_optimizer": true,
    "optimizer": {
        "type": "Adam",
        "params": {
            "lr": 0.0001,
            "weight_decay": 0.01,
            "bias_correction": true,
            "eps": 1e-06
        }
    },
    "gradient_clipping": 1.0,
    "wall_clock_breakdown": true

Confirmed the effectiveness of groupped all-gather ops.
timeline of master branch in forward pass

timeline of this PR in forward pass

Hey @tjruwase I saw the code failed due to stderr: RuntimeError: Tensors must be CUDA and dense. is there anything I need to do for the fix?

Hey @tjruwase I saw the code failed due to stderr: RuntimeError: Tensors must be CUDA and dense, in nv-transformers-v100 section. is there anything I need to do for the fix?

I do see another error message in this test before this dense related one:
RuntimeError: Invalid function argument. Expected parameter `tensor_list` to be of type List[torch.Tensor].

Are any of your new all_gather calls passing a single tensor instead of a list of tensors? I haven't looked over all the proposed code here but maybe the line I added a comment to.

@zarzen, thanks for following up and sorry that I forgot to update you. Yes, I was able to fix the issue. The problem was that 2 zero context objects were constructed along the way and a parameter that was gathered by one context was partitioned in the other. The fix is to avoid multiple zero contexts, but rather to reuse the existing context to register any newly discovered parameter. The core of the fix is here.
We are fortunate that the HF unit tests was able to expose this issue. I will take a closer look at your unit test failures as well.

@zarzen, thanks for following up and sorry that I forgot to update you. Yes, I was able to fix the issue. The problem was that 2 zero context objects were constructed along the way and a parameter that was gathered by one context was partitioned in the other. The fix is to avoid multiple zero contexts, but rather to reuse the existing context to register any newly discovered parameter. The core of the fix is here.
We are fortunate that the HF unit tests was able to expose this issue. I will take a closer look at your unit test failures as well.

Are you refer to this commit, a75e46, for fixing the multi-context issue?
does that mean I can wait #1170 get merged first?


[AMQ-8327] Update Jackson to 2.12.4

This passes all but when Apache Jenkins has a port conflict when testing the assembly:
https://ci-builds.apache.org/blue/organizations/jenkins/ActiveMQ%2FActiveMQ/detail/PR-684/9/pipeline
Still though, re-running to try to get the green checkmark.


Minify source

Minify the source ready for release and restructure such that there is a release folder with just the files necessary to make this work.

Incorrectly labeled the fix from this commit. It actually fixes #5


Certificate missing

I can't install the binaries or build the resources without the files being signed.
Any chance of adding a certificate to the files?

I don’t have any funding for certificates, so at the time being I’m afraid this is not possible.

What's the cost?  Chances are when this terrible meltdown of society is over my company may be able to help.

Update: I convinced my company to sponsor a certificate for Tabular Editor. It should arrive within a few days, at which point I will provide a new build of Tabular Editor with the certificate installed. Big thanks to Kapacity for this. If your company is benefitting from Tabular Editor getting a certificate, please send a happy thought to Kapacity and consider paying our website a visit: www.kapacity.com

Finally! As of 2.9.8 Tabular Editor is now digitally signed. Please let me know if you have any issues!


CC-360 Adding service creation form

addressing issue with jsonform-components lib in Astro project with upgrades
added create service
added template for update service

https://github.com/user-attachments/assets/2230755f-a54f-4726-a4db-d36cff27dd83

still need to hook up the saving part once backend is up and running. but basic ui for now. still working on ironing out some kinks as well as update page.