Support for DSN

I really like your library and also works great, but this lacks support for DSN based reports. do you have any plan for that?

Glad you liked it, we just released this we are using in case it helps other too! I don't have experience working with DSN based connections/reports, but if you can share what settings you use, for sure we can add a CreateDSNConnection in the file https://github.com/gerardo-lijs/CrystalReportsRunner/blob/main/src/LijsDev.CrystalReportsRunner.Core/CrystalReportsConnectionFactory.cs
to fit your needs and of other people using DSN.

Your library is also missing TableLogOnInfo.ApplyLogOnInfo method to set connection to each table in case of DSN of SQL server connection for report

Not sure what issue you are experiencing, but we do set LogOnInfo to every table in every subreport. Take a look in the source file   https://github.com/gerardo-lijs/CrystalReportsRunner/blob/main/src/LijsDev.CrystalReportsRunner/shared/ReportUtils.cs   line 139.
Can you share a sample project with a DSN report that fails so we can try to help you? Otherwise, PR are also open if you can improve the library to be more compatible with the reports you are using.
The library was designed so you can create your own runner -> https://github.com/gerardo-lijs/CrystalReportsRunner/blob/main/docs/custom-runner.md   and just use the NamedPipes communication. This can probably fit any potential user need also.

done and closed


Make hmt-escrow working in the Polygon Mainnet

Short Description
Polygon has a different implementation of the HMToken. It has no bulk related methods. Because of this, hmt-escrow library can't be used in the Polygon, since Escrow contract uses transferBulk method
Main Description
Polygon Mainnet has a standard ERC20 interface and implementation for the HMToken. It is implemented via the standard bridge mechanism and represents a Proxy + Implementation
The Proxy - 0xc748B2A084F8eFc47E086ccdDD9b7e67aEb571BF
The Implementation - 0x4948d86aa5f37fc07e3aa4a6f6f34f740b2752cc
(Actually, the official type for the implementation is not a ERC20, but UChildERC20, which has methods intended for moving assets from Ethereum to Polygon(deposit and withdraw)
Due to this, hmt-escrow can't be used in the Polygon, since Escrow uses transferBulk method during payouts
Expected behavior
hmt-escrow should be available in all EVM compatible blockchains, despite on the type of the HMToken(ERC20 or ERC20 with bulk)
┆Issue is synchronized with this Asana task by Unito

@e271828-
The list you provided looks more like list of solutions, but not a desired behavior. But, I agree that expected behavior description a little bit blurry, let me rephrase it

Details of the solution I propose might be inspected in the PR #295

Solved with #295


Question on watchStuckJobs + promote

I have multiple processes, something like this:

master-web-server (receives jobs from workers2+3, produces jobs for worker4)
worker1 (receives jobs from master, produces jobs for master)
worker2 (receives jobs from master, produces jobs for master)
worker3 (receives jobs from master)

I have been experiencing stuck jobs, and I am combing through the comments/issues to try to figure out how to make this more robust.  I have added graceful shutdown to all of these processes, so that should help.
Other suggestions indicate that you should call jobs.watchStuckJobs(), but should I call that in each process or just the master?
Same question for jobs.promote() -- call in each process or just the master?
I should clarify, it is my understanding that you only need to call promote if you are using delayed jobs, OR if you are using retries (i.e. .attempts(x)) since that uses delays under the covers.

https://github.com/LearnBoost/kue/issues/510#issuecomment-74639328
+
both those methods do a global server side change on redis, so both need to be called in only ONE (i.e. your master) process.
whenever you need a delay (be in delayed or delayed retries) you need promote to be called :)

thanks


bug with find and replace all folder feature

Issue Type: Bug
When i press Ctrl + Shift + H to find something and select replace all, it sometime fail to replace all found text, sometime it replace wrong position, maybe because the position changed since the previous replace of the same file.
VS Code version: Code 1.41.1 (26076a4de974ead31f97692a0d32f90d735645c0, 2019-12-18T14:58:56.166Z)
OS version: Windows_NT x64 10.0.17763

Can you please add concrete steps that help us to reproduce the issue? Thank you!

/needsMoreInfo


[HTTPCLIENT-1952] Allow default User Agent to be disabled

Naive approach works for me. I am not certain whether the @see is necessary. I think not.

It seems you didn't commit all changes (defaultUserAgentDisabled is not used).

@thc202 Thanks, missed that in the interactive add.

One would also need to change the @since if we backport this to 4.5.x.

@michael-o Ideally RequestUserAgent should be removed from the message processing chain entirely but looks good enough to me.


minor haddock issues

Looking at the haddock for Network.Anonymous.Tor:

There's a mention of  detectPort, which is not in the library. Could instead say to use isAvailable to probe for the control ports.
Under "Connect through Tor with explicit port" there's a broken hyperlink on  SocksAddress,
In the "Connect through Tor using control port" example, withSession is called without the control port, so I think that example won't work as written. There are some other examples of withSession that have the same problem.

Also, examples of using accept and mapOnion lack the Maybe ByteString parameter for the private key.


Added waitOn.either for logical workflow path

@RackHD/corecommitters
@pscharla
@VulpesArtificem
There are several highlights:

Task Request comes from:

https://rackhd.atlassian.net/browse/RAC-2043

Since current waitOn can already do:
"trigger task based on both conditions"

This feature is letting workflow engine do:
"trigger task based on one of these conditions ( ie: under waitOn.either) that is fulfilled"

There is also dependency clearing part in on-core:

https://github.com/RackHD/on-core/pull/253
This feature need to be enabled by merging these two PRs.

test this please

BUILD on-tasks #1957 : UNSTABLE

** BUILD build-config-setup  #1280: SUCCESS

** BUILD unit-tests  #11394: SUCCESS

** BUILD SmokeTest-FIT  #230: SUCCESS

** BUILD CIT  #893: SUCCESS
** BUILD FIT  #754: SUCCESS

** BUILD on-taskgraph #2024: UNSTABLE

** BUILD build-config-setup  #1281: SUCCESS
** BUILD unit-tests  #11395: UNSTABLE

** BUILD on-http #3168: SUCCESS

** BUILD build-config-setup  #1282: SUCCESS
** BUILD unit-tests  #11396: SUCCESS

BUILD unit-tests  #11395  Error Logs  ▼Test Name: Graph Library should validate all existing graphs not requiring user input for null values
Error Details: Task.Ucs.Catalog: expected undefined to exist
Stack Trace: AssertionError: Task.Ucs.Catalog: expected undefined to exist

BUILD on-taskgraph #2024  Error Logs  ▼Test Name: Graph Library should validate all existing graphs not requiring user input for null values
Error Details: Task.Ucs.Catalog: expected undefined to exist
Stack Trace: AssertionError: Task.Ucs.Catalog: expected undefined to exist

what is task.ucs.catalog?

test this please

test this please

@zhangy42 : you need first rebase your PR, those code about UCS catalog shouldn't belong to your PR.

@zhangy42 : as for the keyword either, it means one or the other of two: (see http://www.dictionary.com/browse/either), but in our cases, we many define 3 or even more waitOn conditions. so the keyword need be changed.
From my experience on JSON-Schema, I think the word anyOf is suitable here (see https://spacetelescope.github.io/understanding-json-schema/reference/combining.html).

BUILD on-tasks #2013 : UNSTABLE

** BUILD build-config-setup  #1808: SUCCESS

** BUILD unit-tests  #11819: SUCCESS

** BUILD SmokeTest-CIT  #683: SUCCESS

** BUILD SmokeTest-FIT  #489: SUCCESS

** BUILD Install-CentOs-6.5  #474: SUCCESS

** BUILD Install-Ubuntu-14-04  #452: SUCCESS

** BUILD on-taskgraph #2104: SUCCESS

** BUILD build-config-setup  #1809: SUCCESS
** BUILD unit-tests  #11820: SUCCESS

** BUILD on-http #3297: UNSTABLE

** BUILD build-config-setup  #1810: SUCCESS
** BUILD unit-tests  #11821: UNSTABLE

BUILD unit-tests  #11821  Error Logs  ▼Test Name: Http.Api.Files "before all" hook: start HTTP server
Error Details: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.
Stack Trace: Error: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.
Test Name: Http.Api.Versions "before all" hook: start HTTP server
Error Details: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.
Stack Trace: Error: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.

Too many irrelevant changes and conflicts for rebase, making another banch and another PR.


Example not working correctly

Example didn't work at all because of the line p.validate.each { |e| puts e.message }
(undefined method `message' for #Hash:0x007f8dfa269118 (NoMethodError))
After removing that line example.rb runs great except when I open the resulting xlsx in excel I get the following error: 'Excel could not open example.xlsx because some content is unreadable'
And produces the following log file:
    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <recoveryLog xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
      <logFileName>Repair Result to example 05025.xml</logFileName>
      <summary>Errors were detected in file 'Lion:Users:jurriaanpruis:axlsx:example.xlsx'</summary>
      <removedRecords summary="Following is a list of removed records:">
        <removedRecord>Removed Records: View from /xl/worksheets/sheet2.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet3.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet4.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet5.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet6.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet7.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet8.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet9.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet10.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet11.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet12.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet13.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet14.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet15.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet16.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet17.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet18.xml</removedRecord>
        <removedRecord>Removed Records: View from /xl/worksheets/sheet19.xml</removedRecord>
      </removedRecords>
      <repairedRecords summary="Following is a list of repairs:">
        <repairedRecord>Repaired Records: Drawing from /xl/drawings/drawing3.xml (Drawing shape)</repairedRecord>
        <repairedRecord>Repaired Records: Drawing from /xl/drawings/drawing5.xml (Drawing shape)</repairedRecord>
        <repairedRecord>Repaired Records: Drawing from /xl/drawings/drawing6.xml (Drawing shape)</repairedRecord>
      </repairedRecords>
    </recoveryLog>

After repair I also see that the chinese characters are not displayed correctly.

Thank @jurriaan Ill have a look now

Hey mate, is this happening on HEAD:master or something you  are working on?
If you are seeing:
Example didn't work at all because of the line p.validate.each { |e| puts e.message }
(undefined method `message' for #Hash:0x007f8dfa269118 (NoMethodError))
Then there are validation errors in the doc.
Either way , I am going to need to change Package#validate as I have obviously broken backwards compatibility.

ok, the validation errors are caused by myself, I will investigate that ;)

ok, the validation errors in excel are caused by myself, I will investigate that ;) But the  p.validate.each error is happening on master..

And those chinese characters are still being shown like this: http://dump.jurriaanpruis.nl/171792523601.png

@jurriaan
I've just reverted my change to Package#validate.
My bad for breaking the backward compatibility. It should now start showing the message when you run example.rb.
re: The Japanese characters in the sheet name:
What language version are you using?
I am going to close this issue and open another specifically for MBCS in sheet names.


Update xeus-cling version

I guess mime_bundle_repr doesn't work in mybinder because of the version of xeus-cling, what do you think @SylvainCorlay ?

Yes, althoufh we will need to wait for the next xtensor version for the binder to reflect it unless we change the link to point to master.


Using Function components in JSX

Hey @tomtheisen, really like your work on mutraction!
Have a question regarding to usage of Function components, is it real to use that like in react?
const ComponentOne = () => {
    return
One
;
};

const ComponentTwo = (props:{ value: string }) => {
    return
{props.value}
;
};

const App = () => {
    return

        <ComponentOne />
        <ComponentTwo value="Two" />

;
};

Thank you!

Thanks for your interest in mutraction.
In general, instead of invoking a component using <MyComponent value={123} />, you can do it like this.
{ MyComponent({ value: 123 }) }

So your code example could be written this way.
const ComponentOne = () => {
    return
One
;
};

const ComponentTwo = (props:{ value: string }) => {
    return
{props.value}
;
};

const App = () => {
    return

        { ComponentOne() }
        { ComponentTwo({value: "Two"}) }

;
};

Here's a sandbox link
I left a comment on another issue with some of the reasoning for this design.  But briefly, it allows the full use of JS syntax in function invocation, and eliminates special cases when doing the JSX transform.  Subjectively, I find it equally readable.


Facebook Ads Customer match

I'm working with @vpoiesz and Zev on this one!
This action is analogous to the Google Ads Customer Match action, but for Facebook Ads.
Feedback appreciated!

@davidtamaki and I are going to review this and iterate with the consulting partner if needed. Then we'll work with you @phillipperalez for final merge approval and launch prep. We'll circle back when ready!

Updated! Running a final test now to make sure my final cleanup didnt break anything

note that JSONDetailLiteStream seems to always send the full results set, regardless of whether you want to limit to "in table results" or not. Better than the reverse!


optimalPartInfo() default partSize results in running out of memory

Like #404, I'm streaming data which could be over 1GB and trying to use PutObjectStreaming(), but this results in optimalPartInfo(-1) being called and a partSize of 603979776 being returned.
The problem I face is that this partSize is used as, essentially, the size of the read buffer, and I'm running out of memory (I end up with errors like fork/exec ...: cannot allocate memory after the stream completes).
I've tried setting partSize = minPartSize in optimalPartInfo() when the input size is -1, and this solves my memory issue.
Regardless of exactly how it's done, can we get (or choose) a reasonable sized read buffer during PutObjectStreaming() so that we can stream in constant small amount of memory?

(In my test doing minPartSize chunks also resulted in greater speed, so I'm not sure why larger sizes are considered more optimal?)

You should always make sure that the size of the stream is known for AWS S3 APIs to work in optimal manner. If not we fallback to a buffering technique where maximum object chosen is 5TB (lack of knowing file size) - If you do know the file size wrap it around into a struct with Len() function then optimalPartInfo() will calculate the right value @sb10 .
So this is an expected behavior. Also the buffering technique only uses 576MB re-usable buffer are you saying you don't have 1GB > memory on the system where you are using this?

@harshavardhana I don't know the size of the file beforehand; the use case here is creating data that is too big to fit on the local hard drive and instead directly streaming it to S3.
From my brief reading of the code a new ~600MB buffer is created for every call to PutObjectStreaming(), so if someone wanted to stream multiple files simultaneously they'd run out of memory even on large memory systems.
But I'm running out of memory streaming just 1 file on a 4GB system. According to free -h I go from 68MB used to 2.7GB used (~900MB available) during the stream of a 1GB file. I run out of memory during a subsequent unrelated call. This may be due to being forced to buffer the writes while waiting for minio-go to read, and possibly a memory leak somewhere. But this change to minio-go fixes the problem:
$ git diff upstream/master
diff --git a/api-put-object-common.go b/api-put-object-common.go
index 213fc21..fb26417 100644
--- a/api-put-object-common.go
+++ b/api-put-object-common.go
@@ -56,19 +56,23 @@ func isReadAt(reader io.Reader) (ok bool) {
 //  maxMultipartPutObjectSize - 5TiB
 //
 func optimalPartInfo(objectSize int64) (totalPartsCount int, partSize int64, lastPartSize int64, err error) {
-       // object size is '-1' set it to 5TiB.
-       if objectSize == -1 {
-               objectSize = maxMultipartPutObjectSize
-       }
        // object size is larger than supported maximum.
        if objectSize > maxMultipartPutObjectSize {
                err = ErrEntityTooLarge(objectSize, maxMultipartPutObjectSize, "", "")
                return
        }
-       // Use floats for part size for all calculations to avoid
-       // overflows during float64 to int64 conversions.
-       partSizeFlt := math.Ceil(float64(objectSize / maxPartsCount))
-       partSizeFlt = math.Ceil(partSizeFlt/minPartSize) * minPartSize
+       var partSizeFlt float64
+       if objectSize == -1 {
+               // object size is '-1' set it to 5TiB and make our part size the minimum
+               // size, so that our read buffer isn't unnecessarily large
+               partSizeFlt = minPartSize
+               objectSize = maxMultipartPutObjectSize
+       } else {
+               // Use floats for part size for all calculations to avoid
+               // overflows during float64 to int64 conversions.
+               partSizeFlt = math.Ceil(float64(objectSize / maxPartsCount))
+               partSizeFlt = math.Ceil(partSizeFlt/minPartSize) * minPartSize
+       }
        // Total parts count.
        totalPartsCount = int(math.Ceil(float64(objectSize) / partSizeFlt))
        // Part size.

Now, during the stream of a 1GB file I go from 68MB used to 1.3GB used (2.3GB available) during the stream, and don't run out of memory.

@harshavardhana I don't know the size of the file beforehand; the use case here is creating data that is too big to fit on the local hard drive and instead directly streaming it to S3.
From my brief reading of the code a new ~600MB buffer is created for every call to PutObjectStreaming(), so if someone wanted to stream multiple files simultaneously they'd run out of memory even on large memory systems.
But I'm running out of memory streaming just 1 file on a 4GB system. According to free -h I go from 68MB used to 2.7GB used (~900MB available) during the stream of a 1GB file. I run out of memory during a subsequent unrelated call. This may be due to being forced to buffer the writes while waiting for minio-go to read, combined with the data generator holding an input buffer of 1GB in my test case. But this change to minio-go fixes the problem:

It is incorrect to assume a lower size just to make sure that it fits into your memory. I agree that the crash is incorrect and at max we should only be using 600MB max.  Unfortunately on AWS S3 it is not possible to upload a file for which content length is not known.  So we optimize based on available information to provide a best effort mechanism.

From my brief reading of the code a new ~600MB buffer is created for every call to PutObjectStreaming(), so if someone wanted to stream multiple files simultaneously they'd run out of memory even on large memory systems.

I see now you are right and it should be avoided, there should be a fixed buffer which should be re-used. The problem is caused by usage of bytes.Buffer which does 2x allocation growth.  The code can be modified to have a capped behavior.   On the other hand reducing the optimal part size is not acceptable solution.

This is not fixed. It actually got worse (my workaround of calling debug.FreeOSMemory() before doing the next thing no longer works because now it fails directly in the minio-go function call). Tested with latest commit 1a09415eed6025360c2c3142f92d3bcf339f873d .
runtime: out of memory

runtime stack:
runtime.throw(0xb0009d, 0x16)
    /home/ubuntu/go/src/runtime/panic.go:596 +0x95
runtime.sysMap(0xc420d40000, 0x24000000, 0x0, 0xdda7d8)
    /home/ubuntu/go/src/runtime/mem_linux.go:216 +0x1d0
runtime.(*mheap).sysAlloc(0xdc0000, 0x24000000, 0x12000)
    /home/ubuntu/go/src/runtime/malloc.go:440 +0x34e
runtime.(*mheap).grow(0xdc0000, 0x12000, 0x0)
    /home/ubuntu/go/src/runtime/mheap.go:774 +0x71
runtime.(*mheap).allocSpanLocked(0xdc0000, 0x12000, 0xe)
    /home/ubuntu/go/src/runtime/mheap.go:678 +0x421
runtime.(*mheap).alloc_m(0xdc0000, 0x12000, 0x100000000, 0xffffffffffffffff)
    /home/ubuntu/go/src/runtime/mheap.go:562 +0xe2
runtime.(*mheap).alloc.func1()
    /home/ubuntu/go/src/runtime/mheap.go:627 +0x4b
runtime.systemstack(0x7f46357f9e00)
    /home/ubuntu/go/src/runtime/asm_amd64.s:343 +0xab
runtime.(*mheap).alloc(0xdc0000, 0x12000, 0x289010100000000, 0x7f463fbd5110)
    /home/ubuntu/go/src/runtime/mheap.go:628 +0xa0
runtime.largeAlloc(0x24000000, 0xc420282801, 0xc4202827e0)
    /home/ubuntu/go/src/runtime/malloc.go:807 +0x93
runtime.mallocgc.func1()
    /home/ubuntu/go/src/runtime/malloc.go:702 +0x3e
runtime.systemstack(0xc42001f300)
    /home/ubuntu/go/src/runtime/asm_amd64.s:327 +0x79
runtime.mstart()
    /home/ubuntu/go/src/runtime/proc.go:1132

goroutine 3585 [running]:
runtime.systemstack_switch()
    /home/ubuntu/go/src/runtime/asm_amd64.s:281 fp=0xc420282720 sp=0xc420282718
runtime.mallocgc(0x24000000, 0xa52f80, 0xc420282801, 0x437098)
    /home/ubuntu/go/src/runtime/malloc.go:703 +0x753 fp=0xc4202827c8 sp=0xc420282720
runtime.makeslice(0xa52f80, 0x24000000, 0x24000000, 0x8000000, 0x0, 0x0)
    /home/ubuntu/go/src/runtime/slice.go:54 +0x7c fp=0xc420282818 sp=0xc4202827c8
github.com/minio/minio-go.glob..func1(0xdb04c0, 0x0)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-put-object-multipart.go:59 +0x7f fp=0xc420282880 sp=0xc420282818
sync.(*Pool).Get(0xdb04c0, 0xc4200145a0, 0xc420014570)
    /home/ubuntu/go/src/sync/pool.go:144 +0x105 fp=0xc4202828d8 sp=0xc420282880
github.com/minio/minio-go.Client.putObjectMultipartNoStream(0xc4206331c0, 0x5, 0x0, 0x0, 0x0, 0xc4206331c8, 0x10, 0x0, 0x0, 0x0, ...)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-put-object-multipart.go:110 +0x390 fp=0xc4202830a8 sp=0xc4202828d8
github.com/minio/minio-go.Client.putObjectMultipart(0xc4206331c0, 0x5, 0x0, 0x0, 0x0, 0xc4206331c8, 0x10, 0x0, 0x0, 0x0, ...)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-put-object-multipart.go:37 +0x10c fp=0xc420283378 sp=0xc4202830a8
github.com/minio/minio-go.Client.putObjectCommon(0xc4206331c0, 0x5, 0x0, 0x0, 0x0, 0xc4206331c8, 0x10, 0x0, 0x0, 0x0, ...)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-put-object.go:203 +0x305 fp=0xc420283610 sp=0xc420283378
github.com/minio/minio-go.Client.PutObjectWithProgress(0xc4206331c0, 0x5, 0x0, 0x0, 0x0, 0xc4206331c8, 0x10, 0x0, 0x0, 0x0, ...)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-put-object.go:184 +0x18f fp=0xc4202837a8 sp=0xc420283610
github.com/minio/minio-go.Client.PutObjectStreaming(0xc4206331c0, 0x5, 0x0, 0x0, 0x0, 0xc4206331c8, 0x10, 0x0, 0x0, 0x0, ...)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-put-object-streaming.go:31 +0xf8 fp=0xc420283938 sp=0xc4202837a8
github.com/[mycode].UploadData(0xc420016960, 0xd835e0, 0xc420078138, 0xc4202094e0, 0x13, 0x487703, 0x59888cac)
    /home/ubuntu/work/src/github.com/[mycode].go:290 +0x147 fp=0xc420283bb0 sp=0xc420283938
[...]

[..., lots and lots of:]

goroutine 1904 [select]:
github.com/minio/minio-go.Client.GetObject.func1(0xc42045b500, 0xc42045b5c0, 0xc42045be00, 0xc42067c920, 0xc4205928c0, 0xc42067c930, 0xc420618fc0, 0xc42001c070, 0x4, 0xc420221440, ...)
    /home/ubuntu/work/src/github.com/minio/minio-go/api-get-object.go:89 +0x1b18
created by github.com/minio/minio-go.Client.GetObject
    /home/ubuntu/work/src/github.com/minio/minio-go/api-get-object.go:233 +0x301

Where mycode line 290 is:
_, err := a.client.PutObjectStreaming(a.bucket, dest, data)

This is not fixed. It actually got worse (my workaround of calling debug.FreeOSMemory() before doing the next thing no longer works because now it fails directly in the minio-go function call). Tested with latest commit 1a09415 .

The memory is protected by pooling the buffers you can't use debug.FreeOSMemory() which is a hack. The memory cannot be capped in the same way in traditional C programs.  There is no way to optimize this on our end since AWS S3 API doesn't provide an API where content length is not provided it means we need to know the length of the we are going to upload and we have to upload the exact size, whatever we are doing right now in this library is sort of handling a scenario which no other AWS SDKs handle. We tried using temporary files but even that has challenges on systems with low on disk space.
I would suggest an architecture change on your end from what you are doing right now and not provide minio-go a stream with no length.  If you do provide final length then we do know how much memory to allocate optimally.  You can implement a custom io.Reader - which implements a Len() or  Size() returns the size of the io.Reader underneath simplifies the behavior here.

I'll look in to exactly what minio-go is doing tomorrow, but what stops you from uploading in user-defined or small chunks? If you only had a few 64MB buffers, I wouldn't run out of memory.
Is there a provable benefit to having buffers larger than this size?

I'll look in to exactly what minio-go is doing tomorrow, but what stops you from uploading in user-defined or small chunks? If you only had a few 64MB buffers, I wouldn't run out of memory.

APIs need to be changed to address this and this would come across as a low-level API. Now if you do know that 64MB is good enough per chunk - then you are sort of aware of the final length of the object that you are going to upload. So i would suggest wrap the reader and implement Size() function which would then by used internally to calculate the optimal chunk.
If you wish to use low level APIs then you can simply use http://godoc.org/github.com/minio/minio-go#Core - here chunking and memory management is all yours - you can decide when to do multipart etc.

Is there a provable benefit to having buffers larger than this size?

The API needs to cater for all use cases, if it supports no-length io.Reader then it should support uploading till 5TB (maximum allowable object size for AWS S3) in such a case 500MB+ per part is the most optimal number based on the 10,000 part limit of AWS S3.

I have no idea about the file size; files are being streamed to me externally.
In terms of catering for all use cases, how about limiting the size and number of buffers in the pool based on available memory? I feel like it's more useful to avoid crashing than it is to be able to upload a 5TB file. After all, you can't upload a 5TB file if you crash.
The problem I was having earlier was when I only had a few hundred MB of memory free on my machine. The very first attempt to create the ~600MB buffer failed.
But on a fresh 4GB machine with no other user processes running:
$ free -m
                  total        used        free      shared  buff/cache   available
Mem:           3951          47        3847           5          56        3746
Swap:             0           0           0

And having altered minio-go like this:
diff --git a/api-put-object-multipart.go b/api-put-object-multipart.go
index 6e0015a..1a4720a 100644
--- a/api-put-object-multipart.go
+++ b/api-put-object-multipart.go
@@ -24,6 +24,7 @@ import (
        "io/ioutil"
        "net/http"
        "net/url"
+       "log"
        "sort"
        "strconv"
        "strings"
@@ -56,7 +57,9 @@ func (c Client) putObjectMultipart(bucketName, objectName string, reader io.Read
 var bufPool = sync.Pool{
        New: func() interface{} {
                _, partSize, _, _ := optimalPartInfo(-1)
+               log.Printf("will create a new buffer of size %d\n", partSize)
                b := make([]byte, partSize)
+               log.Println("success!")
                return &b
        },
 }
diff --git a/api-put-object-streaming.go b/api-put-object-streaming.go
index 0d4639e..041fae4 100644
--- a/api-put-object-streaming.go
+++ b/api-put-object-streaming.go
@@ -20,6 +20,7 @@ import (
        "fmt"
        "io"
        "net/http"
+       "log"
        "sort"
        "strings"

@@ -28,6 +29,7 @@ import (

 // PutObjectStreaming using AWS streaming signature V4
 func (c Client) PutObjectStreaming(bucketName, objectName string, reader io.Reader) (n int64, err error) {
+       log.Printf("PutObjectStreaming called for %s\n", objectName)
        return c.PutObjectWithProgress(bucketName, objectName, reader, nil, nil)
 }

Trying to upload 2 (~10 byte) files (sequentially) normally results in:
2017/08/08 08:55:52 PutObjectStreaming called for wr_tests/a.file
2017/08/08 08:55:52 will create a new buffer of size 603979776
2017/08/08 08:55:53 success!
2017/08/08 08:55:53 PutObjectStreaming called for wr_tests/mvsubdir/file
2017/08/08 08:55:53 will create a new buffer of size 603979776
2017/08/08 08:55:54 success!

But sometimes, just running the same code over and over, I get:
2017/08/08 08:56:07 PutObjectStreaming called for wr_tests/a.file
2017/08/08 08:56:07 will create a new buffer of size 603979776
2017/08/08 08:56:08 success!
2017/08/08 08:56:08 will create a new buffer of size 603979776
signal: killed

I don't really understand where the kill signal is coming from, nor why a second buffer gets created before I see the log call for trying to stream the second file.
And, since I'm uploading files sequentially, why doesn't the first buffer get reused? Isn't that the point of having a pool? Why do I need to spend 1.2GB of memory to upload ~20bytes of data?

Here's code that triggers the issue:
package main

import (
	"github.com/minio/minio-go"
	"io"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
)

func main() {
	tmpdir, err := ioutil.TempDir("", "upload_testing")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(tmpdir)

	upFile1 := filepath.Join(tmpdir, "file1")
	upBytes := []byte("upfile\n")
	err = ioutil.WriteFile(upFile1, upBytes, 0644)
	if err != nil {
		log.Fatal(err)
	}
	upFile2 := filepath.Join(tmpdir, "file2")
	err = ioutil.WriteFile(upFile2, upBytes, 0644)
	if err != nil {
		log.Fatal(err)
	}

	client, err := minio.NewV2("cog.sanger.ac.uk", os.Getenv("AWS_ACCESS_KEY_ID"), os.Getenv("AWS_SECRET_ACCESS_KEY"), true)
	// client.TraceOn(nil)

	bucket := "sb10"

	upload(client, bucket, upFile1)
	upload(client, bucket, upFile2)
}

func upload(client *minio.Client, bucket, path string) {
	rpipe, wpipe := io.Pipe()

	go func() {
		// in my real use case I'm not opening a file, I am streamed chunks of
		// data; simulate that here
		f, err := os.Open(path)
		if err != nil {
			log.Fatal(err)
		}
		defer f.Close()

		b := make([]byte, 7)
		_, err = f.Read(b)
		if err != nil {
			log.Fatal(err)
		}
		wpipe.Write(b)
		wpipe.Close()
	}()

	log.Printf("upload of %s...\n", path)
	_, err := client.PutObjectStreaming(bucket, filepath.Base(path), rpipe)
	if err != nil {
		log.Fatal(err)
	}
	log.Println("upload succeeded")
	client.RemoveObject(bucket, filepath.Base(path))
}

If I run go run -tags netgo -race main.go then I mostly get:
2017/08/08 09:57:14 upload of /tmp/upload_testing054851088/file1...
2017/08/08 09:57:14 PutObjectStreaming called for file1
2017/08/08 09:57:14 will create a new buffer of size 603979776
2017/08/08 09:57:15 success!
2017/08/08 09:57:15 upload succeeded
2017/08/08 09:57:15 upload of /tmp/upload_testing054851088/file2...
2017/08/08 09:57:15 PutObjectStreaming called for file2
2017/08/08 09:57:15 will create a new buffer of size 603979776
signal: killed

But sometimes get:
2017/08/08 09:57:49 upload of /tmp/upload_testing028591414/file1...
2017/08/08 09:57:49 PutObjectStreaming called for file1
2017/08/08 09:57:49 will create a new buffer of size 603979776
2017/08/08 09:57:50 success!
2017/08/08 09:57:50 will create a new buffer of size 603979776
signal: killed

So it gets killed 100% of the time, but sometimes creates the second buffer early. Why is it creating a second buffer?

(It seems to always work fine without -race, but I need my tests to pass cleanly with -race enabled.)

It looks like the first and single call to bufPool.Get() in putObjectMultipartNoStream() can result in bufPool's New function being called twice. Sequential calls also result in the New function being called each time, despite the bufPool.Put() calls.
From the docs for sync.Pool:

Get may choose to ignore the pool and treat it as empty. Callers should not assume any relation between values passed to Put and the values returned by Get.

Maybe this behaviour isn't so surprising. So I'd suggest that sync.Pool is not the ideal way to implement this.

I was wrong about a single bufPool.Get() resulting in bufPool's New function being called twice. A single putObjectMultipartNoStream() call results in multiple bufPool.Get() calls, the last of which at EOF currently doesn't result in a Put(), which is a bug.
Fixing this doesn't solve my issue though, since sync.Pool still creates unnecessary buffers. Please see my pull request for a version that works with the above code.


Review runbooks for on-call readiness

This is about going through some scenarios of what might break while on-call and ensuring I have all the knowledge (and tools) to be able to handle incidents while on-call.
Example scenarios:

Graphs drop to zero in the weekend, what do I do?

This has been done.


Dragging outside of the color picker stops updating the color

In react-color if you click and drag outside of the color picker, it still updates the color.
In color-picker the updating stops and the picker freezes.
Example:
https://www.loom.com/share/8bc6a40a304944578008b01649638142

I believe it needs to be updated to use document instead of a reference to the picker.
https://github.com/hello-pangea/color-picker/blob/main/packages/color-picker/src/components/common/Saturation.tsx#L45
The calculate change method also needs to be updated.
https://github.com/hello-pangea/color-picker/blob/main/packages/color-picker/src/helpers/saturation.js


⚠ Bofrost.com has degraded performance

In e9efdb7, Bofrost.com (https://www.bofrost.com) experienced degraded performance:

HTTP code: 200
Response time: 703 ms

Resolved: Bofrost.com performance has improved in 117a4b2 after 9 minutes.


Short solution needed: "PHP PDO connect to Mysql" (php-pdo)

Please help us write most modern and shortest code solution for this issue:
PHP PDO connect to Mysql (technology: php-pdo)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

Connect to Mysql with PHP PDO


Not on PYPI?

Hi. This looks like a nice library, but I couldn't pip install rfeed. Publishing on PyPI is really easy, see https://python-packaging-user-guide.readthedocs.io/en/latest/distributing/

I'm published this package on PyPI: https://pypi.python.org/pypi/rfeed with changes. see: https://github.com/egorsmkv/rfeed

What is the "official" repository for this project? @egorsmkv or @svpino ?
Looks like @egorsmkv is doing the releases on PyPI. I'd like to fill an issue and probably a pull request afterwards. @egorsmkv repository does not accept issues though.
Can you please help me?

@rafaelbco official repo is @svpino https://github.com/svpino/rfeed
@rafaelbco I'm opened issues in my repo.

So, it seems @svpino  is the official repo but I'm better off opening issues and PRs ate @egorsmkv because PyPI releases are made based on that repo. Am I right?

@rafaelbco Yes, you are right.

Since @egorsmkv has archived his fork, and development has been happening here again, perhaps PyPI should be using this repo instead?

Just do this:
pip install --upgrade https://github.com/svpino/rfeed/tarball/master


iPad pinch zoom conflict between iOS and Safari, hackish fix

Describe the bug
When you pinch to zoom in/out the timeline in Safari on iPad, the whole webpage zooms in and out as well as the timeline trying to zoom. The OS pinch is in conflict with the UI element in the webpage. Both want to do different things with the same pinch event.
To Reproduce
Use two fingers to pinch/expand the timeline on an iPad. You will see the whole page grow or shrink, which is a feature of iOS to zoom in on a webpage.
Expected behavior
I expect the timeline to preventDefault behaviour of iOS and just shrink or expand the timeline without scrolling the page or changing its scale.
Screenshots
—
Library Version
0.23
iOS 12.2, iPad (6th generation with pencil support)
Additional context
Here is a hack using Hammerjs that fixes the problem, but a needs to be done the React way with refs instead of looking up the item in the DOM.
pseudo code, may need to wait for DOM to be rendered even though it mounted because this is a hackish solution, not the React way
  ComponentDidMount(){
        // the element you want to attach to
        const myElement = document.getElementById('My-timeline')
        // create a new hammer object, setting "touchAction" ensures the user can still scroll/pan
        const hammertime = new Hammer(myElement, {
          prevent_default: false,
          touchAction: "pan"
        });

        // pinch is not enabled by default in hammer
        hammertime.get('pinch').set({
          enable: true
        })

        // name the events you want to capture, then call some function if you want and most importantly, add the preventDefault to block the normal pinch action
        hammertime.on('pinch pinchend pinchstart doubletap', (e)=> {
          console.log('captured event:', e.type)
          e.preventDefault()
        })
}

…

  <Timeline …/>

solution partially taken from here https://stackoverflow.com/questions/37808180/disable-viewport-zooming-ios-10-safari

Here is the proper way to do it in React using React.createRef()
ComponentDidMount(){

  this.timelineRef = React.createRef()

setTimeout(()=>{ // This is required but is OK to put it at zero seconds, lets DOM render and set ref
        const hammertime = new Hammer(this.timelineRef.current, {
          prevent_default: false,
          touchAction: "pan"
        })

        hammertime.get('pinch').set({ enable: true })

        hammertime.on('pinch pinchend pinchstart doubletap', (e)=> {
          console.log('captured event:', e.type)
          e.preventDefault()
        })
}, 0)
…

      <Timeline … />


Configure error in node-gyp

Node Version: v10.14.2
Platform: WIndows 10
Compiler:
Module:

Verbose output (from npm or node-gyp):ERR! stack Error: Command failed: C:\Users\suvsingh\AppData\Local\Programs\Python\Python37-32\python.EXE -c import sys; print "%s.%s.%s" % sys.version_info
D:\UI\_build\node_modules\.bin>node-gyp rebuild --verbose
gyp info it worked if it ends with ok
gyp verb cli [ 'C:\\Program Files\\nodejs\\node.exe',
gyp verb cli   'D:\\UI\\_build\\node_modules\\node-gyp\\bin\\node-gyp.js',
gyp verb cli   'rebuild',
gyp verb cli   '--verbose' ]
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| win32 | x64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb check python checking for Python executable "python2" in the PATH
gyp verb `which` failed Error: not found: python2
gyp verb `which` failed     at getNotFoundError (D:\UI\_build\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (D:\UI\_build\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (D:\UI\_build\node_modules\which\which.js:80:29)
gyp verb `which` failed     at D:\UI\_build\node_modules\which\which.js:89:16
gyp verb `which` failed     at D:\UI\_build\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at D:\UI\_build\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:154:21)
gyp verb `which` failed  python2 { Error: not found: python2
gyp verb `which` failed     at getNotFoundError (D:\UI\_build\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (D:\UI\_build\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (D:\UI\_build\node_modules\which\which.js:80:29)
gyp verb `which` failed     at D:\UI\_build\node_modules\which\which.js:89:16
gyp verb `which` failed     at D:\UI\_build\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at D:\UI\_build\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:154:21) code: 'ENOENT' }
gyp verb check python checking for Python executable "python" in the PATH
gyp verb `which` succeeded python C:\Users\suvsingh\AppData\Local\Programs\Python\Python37-32\python.EXE
gyp ERR! configure error
gyp ERR! stack Error: Command failed: C:\Users\suvsingh\AppData\Local\Programs\Python\Python37-32\python.EXE -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                                ^
gyp ERR! stack SyntaxError: invalid syntax
gyp ERR! stack
gyp ERR! stack     at ChildProcess.exithandler (child_process.js:294:12)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at maybeClose (internal/child_process.js:962:16)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:251:5)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "D:\\UI\\_build\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild" "--verbose"
gyp ERR! cwd D:\UI\_build\node_modules\.bin
gyp ERR! node -v v10.14.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

I have the same issue with python 3.6 installed.
The code for version check uses print statement, which does not exist in python 3.
In python 3, there's only print function.
change following line (in node_modules/node-gyp/lib/configure.js)
var args = ['-c', 'import sys; print "%s.%s.%s" % sys.version_info[:3];']
to this (add parentheses)
var args = ['-c', 'import sys; print ("%s.%s.%s" % sys.version_info[:3]);']

Python 3 is not supported, use Python 2.x for node-gyp

Thanks a lot guys.
I reverted to Python 2.7, and then moved ahead with the work.


Autocrafting causes server to slow down then stop.

Issue description:
Server goes to 1 tps then watchdog kicks in and reboots it.
What happens:
Server starts to crawl then watchdog kicks in and crashes the server.
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
What you expected to happen:
Steps to reproduce:

Run a large server with a lot of people playing on it.

...
Version (Make sure you are on the latest version before reporting):

Minecraft: 1.10.2
Forge:  <IP_ADDRESS>1
Refined Storage:refinedstorage{1.2.8} [Refined Storage] (refinedstorage-1.2.8.jar)

Does this issue occur on a server? Yes
If a (crash)log is relevant for this issue, link it here:
https://gist.github.com/ProsperCraft/8d9ff185aa207fc11ae55e2e1b66938f

https://gist.github.com/ProsperCraft/11d8e6875ba9d56e27338d8dfc4226f2
Happened again same version.

Can you run some profiling? See #523 (comment)

Try this please:
https://refinedstorage.raoulvdberge.com/wiki/profiling

Likely related to #678

Fixed


Build error testing on Emulator

Hey there -
iOS is working great. But I'm hitting issues building successfully.  I've followed the troubleshooting guide, but seems I'm missing something.
Here is the error:
Execution failed for task ':react-native-maps:processReleaseResources'.
:react-native-maps:processReleaseResources[path to project]/node_modules/react-native-maps/android/lib/build/intermediates/exploded-aar/com.android.support/appcompat-v7/23.0.1/res/drawable-v23/abc_control_background_material.xml:18:24-59 : No resource found that matches the given name (at 'color' with value '@color/abc_color_highlight_material')

> com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command '/Users/kanecochran/Library/Android/sdk/build-tools/23.0.3/aapt'' finished with non-zero exit value 1

I've confirmed I have the SDKs installed, cleaned cache, but still here.
RN 0.30 and RN Maps 0.7.1 developing on Mac.
Thanks!

Can you see if this still is a problem with 0.8.0?

Works great for me.  Thanks guys!!

On Aug 31, 2016, at 1:07 PM, Spike Brehm<EMAIL_ADDRESS>wrote:
Can you see if this still is a problem with 0.8.0 https://github.com/lelandrichardson/react-native-maps/releases/tag/v0.8.0?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/lelandrichardson/react-native-maps/issues/469#issuecomment-243832515, or mute the thread https://github.com/notifications/unsubscribe-auth/AS0VZ0MI7SVYrzHnFgKTmgOK4D3hQPhbks5qlbTagaJpZM4Jk6TC.


Consider generating encoders for output args

The following use case:
 val nginxService = Service(
    "nginx",
    ServiceArgs(
      spec = ServiceSpecArgs(
        selector = labels,
        `type` = k8s.core.v1.enums.ServiceSpecType.LoadBalancer,
        ports = List(...)
      ),
      metadata = ObjectMetaArgs(...)
    )
  )

  Stack.exports(
    nginxUrl = nginxService.status.loadBalancer
  )

Requires the following workaround:
  given besom.types.Encoder[besom.api.kubernetes.core.v1.outputs.PortStatus] = besom.internal.Encoder.derived
  given besom.types.Encoder[besom.api.kubernetes.core.v1.outputs.LoadBalancerIngress] = besom.internal.Encoder.derived
  given besom.types.Encoder[besom.api.kubernetes.core.v1.outputs.LoadBalancerStatus] = besom.internal.Encoder.derived

To get this output from pulumi:
...
Outputs:
    nginxUrl           : {
        ingress: [
            [0]: {
                hostname: "localhost"
            }
        ]
    }
...

Don't we generate that already? There are two options if we don't:
a) add that in codegen and look how much large packages swell, I guess it might be a non trivial amount
b) provide an inline given for Encoders in exports that reach user's code and therefore allow user to derive Encoder on demand
Option A has the benefit of us knowing on package publishing level that something is no yes with caveat of swollen packages.
Option B has the benefit of smaller package size with caveat of missing given errors exposed to the user if something can't be derived.
There's a middle ground option of verifying that we can derive all Encoders for all output args in some kind of post-build test stage (would require two stage generation in codegen, one package would be the core package that we publish, second would depend on first and just verify that we can derive Encoders for all output args so compile failure would prevent publish of first package).

We have encoders for input args but not for output args, only decoder there.
This works OOTB in other implementations, so my expectation, if I was a use, was to be of parity.
I agree this adds code to providers and technical limitations of that needs to be taken into account.


PageCustomization intellisense not picking up actionref keywords

1. Describe the bug
Intellisense does not show addfirst or other add..-keywords

But we can type it on our own and the rest of intellisense is working

5. Versions:

AL Language: v10.0.687650
Visual Studio Code: 1.72.0

The fix for this issue has been checked in to the master branch. It will be available in the bcinsider.azurecr.io/bcsandbox-master Docker image starting from platform build number 22.0.50101.0 and VS Code Extension Version 11.0.716839.
If you don’t have access to these images you need to become part of the Ready2Go program: aka.ms/readytogo
For more details on code branches and docker images please read:
https://blogs.msdn.microsoft.com/nav/2018/05/03/al-developer-previews-multiple-releases-and-github/
https://freddysblog.com/2020/06/25/working-with-artifacts/


Alter some null checks

This MR is not very important. It is mainly a kind of thinking aloud about how to go about nullability.
I checked all the calls to void NavigateTo(MRect) in the samples. What I do now is add the ! when doing the Grow. This will result in an crash at that point. And I think that is a good thing. The Home method is called later on, and that would make it harder to find the bug.
There should be some form of try catch around creating the sample map so we can show the error to the user.

@inforithmics What do you think of this?


CAMEL-XXX Update documentation for SASL_SSL kafka security protocol - camel-3.x

Description
cherry pick https://github.com/apache/camel/pull/9214
Target

[ ] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[ ] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[ ] I checked that each commit in the pull request has a meaningful subject line and body.

[ ] I formatted the code using mvn -Pformat,fastinstall install && mvn -Psourcecheck

@orpiske what about 3.20.x branch ?

@orpiske what about 3.20.x branch ?

That one as well


Restrict coercion of a whole expression

While we allow coercion within an expression according to XPath, we do
not want to allow coercion to the expected expression type. For example,
if the expected type of an expression is an xs:string, we do not want to
coerce { 5 } to the string "5"--the expression should instead be { "5" }
or { xs:string(5) }. This limits coercion to only similar types.
Otherwise, the expected type of an expression must much the actual
expression result.
DAFFODIL-2021

Need to add negative tests showing coercion errors.

Came back to this to add some negative tests, and I dont' think this is really the complete solution. This change still allows coercion in the case where the target type and the expression result type are both numeric. This is clearly wrong in the case of decimal numbers, for example:
<xs:element name="a" type="xs:int" dfdl:inputValueCalc="{ 2.5 }" />

In this case, we really do not want to coerce 2.5 to an int, but this change would coerce since both are numerics.  An alternative might be to require that types match exactly, but then usability suffers. For example:
<xs:element name="a" type="xs:int" dfdl:inputValueCalc="{ 2 }" />

In this case, the literal 2 is technically an xs:integer so the result type and target type do not match. So if we were strict about it, this simple expression requires the use of xs:int(2). That's not terrible, but there's a good chance we would probably break a lot of schemas.
Perhaps a happy medium is to allow coercion between integer-like things? So we would automatically coerece an xs:integer to an xs:int (as long as the value was in range), but would not coerece 2.5 to 2, for example. An odd side effect of this is that one could know do this:
<xs:element name="a" type="xs:int" dfdl:inputValueCalc="{ xs:integer(2) }" />

And we would happily coerce that, even though it seems like maybe the user intended something else since types are explicitly provided and differ. Perhaps a better solution exists? I think it would be difficult track track when types are explicit and only check in those cases, so maybe this edge case isn't worth solving.

In what way is the expression 2 not an xs:int ? Our DPath compiler might choose to make all integers xs:integer and represent them as bignums at runtime, but that's perhaps just arbitrary.
2 fits within the range of every signed and unsigned integer decimal and floating point type.  From the type-system perspective it is of the required type. It should be accepted for any integer type at least.
The usual approach is that conversions that preserve information content perfectly are done automatically within numeric types, but those that involve rounding/truncation are not.

According to w3c XPath, integer literals are alwyas xs:integer and then coerecion happens as needed. But this change is to disable coerecion of expression result. So technical 2 isn't an xs:int. Note that Daffodil does do some stuff to coerece literals down to int's' or long's (though I think technically this isn't spec compliant?), so 2 is probably considered an xs:int. But for example, we never coerce literals to xs:short's, so if you had something like this, it would fail without coerecion:
<xs:element name="a" type="xs:short" dfdl:inputValueCalc="{ 2 }" />

The suggestion of allowing coerence between types only when no rounding/truncation should occur seems reasonable. I'll try it out.


Sequential-dependent codegen breaks with output-file level documents

Describe the bug
The following config works:
schema: schema.graphql
documents: document.graphql
generates:
  types.ts:
    plugins:
      - typescript
      - typescript-operations
  resolvers.ts:
    documents: "./types.ts"
    plugins:
      - typescript
      - typescript-resolvers

The following config does not work:
schema: schema.graphql
generates:
  types.ts:
    documents: document.graphql
    plugins:
      - typescript
      - typescript-operations
  resolvers.ts:
    documents: "./types.ts"
    plugins:
      - typescript
      - typescript-resolvers

And produces this error:
Found 1 error

  ✖ resolvers.ts
    Error:
          Unable to find any GraphQL type definitions for the following pointers:

              - ./types.ts

        at prepareResult (/sandbox/node_modules/@graphql-tools/load/index.cjs.js:590:15)
        at loadTypedefs (/sandbox/node_modules/@graphql-tools/load/index.cjs.js:555:12)
    Error:
          Unable to find any GraphQL type definitions for the following pointers:

              - ./types.ts

        at prepareResult (/sandbox/node_modules/@graphql-tools/load/index.cjs.js:590:15)
        at loadTypedefs (/sandbox/node_modules/@graphql-tools/load/index.cjs.js:555:12)

Yet it works fine if you remove the resolvers.ts definition:
schema: schema.graphql
generates:
  types.ts:
    documents: document.graphql
    plugins:
      - typescript
      - typescript-operations

By moving documents to types.ts, the resolvers.ts definition suddenly thinks that types.ts is invalid, even though it is valid.
To Reproduce
Steps to reproduce the behavior:
https://codesandbox.io/s/delicate-butterfly-2f9wz?file=/codegen.yml
Expected behavior
Behavior should be identical for both configurations, i.e. it should generate the code normally.

FYI in my local environment I can work around this by commenting out the "resolvers.ts" section to first generate the types.ts, and then generating the resolvers.ts after. For some reason this workaround doesn't work in the sandbox environment.

@warent can you please explain the motivation behind using documents: "./types.ts" ? This file will contain only TypeScript types for base types (typescript plugin) and typescript operations types (typescript-operations plugin, based on the documents section).
Also, why do you need to load GrpahQL operations for your 2nd output file that creates resolvers signature? It seems like a misuse of codegen.

The example I provided is not telling the full story. It looks more like this:
schema: schema.graphql
generates:
  types.ts:
    documents: document.graphql
    plugins:
      - typescript
      - typescript-operations
      - typescript-react-apollo
  resolvers.ts:
    documents: "./types.ts"
    plugins:
      - typescript
      - typescript-resolvers

i.e. we're using types.ts for frontend code with typescript-react-apollo, and resolvers.ts for backend Node.js code.
Should we be doing something else?

Also note we're using a third party library that utilizes the resolvers.ts file

Got it, so what's the reason for passing document for the 2nd output file?
schema: schema.graphql
generates:
  types.ts:
    documents: document.graphql
    plugins:
      - typescript
      - typescript-operations
      - typescript-react-apollo
  resolvers.ts:
    documents: "./types.ts"                                     ##   <<< this one doesnt make sense here
    plugins:
      - typescript
      - typescript-resolvers

I'm asking because both typescript and  typescript-resolvers doesn't have anything to do with document  - both types are dealing with schema types, and not your operations.
Your setup seems correct except for the line I specified (it can be removed). Putting the schema on the top level is great (so it will be available for both output files), and the document only for the one that generates operation types is also a good direction.

I'm not sure where I got the idea that I had to do that. Thank you for the correction!


refactor(SourceFileReporter): Add new addMessage signature

LGTM and I'm fine with using this for the time being.
I still find the signature somewhat complicated. It would be nice if we could limit it to a few possible parameters, maybe generalizing some of them to fit all possible messages. But I see that this might not be possible given how different some of the messages are.
An alternative to this would be to have dedicated functions per message type, each with statically typed parameters. I don't think this would make it easier for the consuming modules tough.


install<EMAIL_ADDRESS>tip Cannot convert undefined or null to object

I npm install auto upgrade to nuxt_webpack@2.10.2,and I  npm run dev than tip error Cannot convert undefined or null to object,does any one meat this error?

This question is available on Nuxt community (#c9971)

你解决了吗兄戴？

有解决么?


Center align

#47

Should we just close this given all the commits here are captured by #57 ?

oops yes I close it


Add custom lldinit for a scheme

Adds support for customLLDBInit, added to XcodeProj in https://github.com/tuist/XcodeProj/pull/553

Nice! Can you also add this to the fixture tests?

@brentleyjones, sure, added one.


fix FileMeta's data file name of Parquet

What changes were proposed in this pull request?
to make DataFileName field in meta file of Parquet file format only store data file's name, excluding path.
How was this patch tested?
unit test

code is good to me. But what's the purpose @Moonlit-Sailor ?

@linhong-intel when the data format is Oap, the filename appears to only contain the filename, while for parquet, the filename contains the full uri. Since the meta file has already contained the path to the folder, we only need to put the filename in filemeta.


Bug in new Salsa pretty-printed stack traces: wrong stack traces

The salsa pretty stack trace is currently off-by-one, skipping the first function called and duplicating the last one:
julia> using Salsa

julia> @derived f(rt, x) = g(rt, x)
f (generic function with 1 method)

julia> @derived g(rt, x) = h(rt, x)
g (generic function with 1 method)

julia> @derived h(rt, _) = error("OH NO")
h (generic function with 1 method)

julia> f(Runtime(), 1)
ERROR: SalsaWrappedException: Error encountered while executing Salsa derived function:
OH NO

------ Salsa Trace -----------------
[1] @derived f(::Runtime, 1::Any)
[2] @derived g(::Runtime, 1::Any)
[3] @derived h(::Runtime, 1::Any)
------------------------------------

Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] %%__user_h(::Salsa._TracingRuntime{Salsa.EmptyContext,Salsa._DefaultSalsaStorage.DefaultStorage}, ::Int64) at ./REPL[16]:1
 [3] invoke_user_function(::Salsa._TracingRuntime{Salsa.EmptyContext,Salsa._DefaultSalsaStorage.DefaultStorage}, ::Salsa.DerivedKey{typeof(h),Tuple{Any}}, ::Int64) at /Users/daly/julia-dev/Salsa/src/Salsa.jl:612
...

Also, I feel like we're maybe printing the stack backwards? I'm not sure what's best - it kind of makes sense to print top-down since that's like the order of the demand, but usually stack traces are printed the other way. I kind of like though that you can just scroll to the middle and read both the pretty stack trace for the current frame and the julia stack frame of the failure, so that's kind of nice?

CC: @comnik for your opinion on the printing order...
Regarding the best order to print them in, it's not hard to change, we just need to pick. Some reasonable options:

Currently (demand-first):------ Salsa Trace -----------------
[1] @derived f(::Runtime, 1::Any)
[2] @derived g(::Runtime, 1::Any)
[2] @derived h(::Runtime, 1::Any)
------------------------------------

Same order reversed numbering:------ Salsa Trace -----------------
[3] @derived f(::Runtime, 1::Any)
[2] @derived g(::Runtime, 1::Any)
[1] @derived h(::Runtime, 1::Any)
------------------------------------

Inside out (standard stacktrace order):------ Salsa Trace -----------------
[1] @derived h(::Runtime, 1::Any)
[2] @derived g(::Runtime, 1::Any)
[3] @derived f(::Runtime, 1::Any)
------------------------------------

@rai-nhdaly Let's run with the order we have for a bit and figure out how it works in practice?


authorization endpoint - response_mode parameter

Hi, how can I set the response_mode parameter to query for the /authorization endpoint call?
(See: https://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#ResponseModes)
I didn't see an option to configure this in the configuration object on this page: https://github.com/AxaFrance/oidc-client/tree/main/packages/react-oidc
Thank you.

Hi @kramer99 thank you for your issue.
I am not sure it is implemented, i will read it more in détail tomorrow.

Hi @kramer99 , i think you can already do that by using extras parameters.
With {response_mode:query}
I am from my mobile i will give you à better answer when i will be front of a computer.

Yes, you can set it through extras:
extras: {
	response_mode: 'query',
},

However, according to the spec, your auth provider should use it by default for the response_type=code:

For purposes of this specification, the default Response Mode for the OAuth 2.0 code Response Type is the query encoding.

Thank you both. I'll close this now.


Error deploying Hashed Collator to k8s

I am trying to deploy the feature/hashed-chain-spec branch to k8s. I am using the same configuration that was used for MD5 Network.
The errors are:
Node scale up in zones us-east4-c associated with this pod failed: GCE out of resources. Pod is at risk of not being scheduled.

and
0/2 nodes are available: 2 Insufficient cpu.

I attempted to add a new Node Pool using Google Cloud, but it didn't have any impact.
I also tried changing the CPU and RAM between 3G and 1.5 CPU to 16G and 4 CPU with no impact.
Command to reproduce is:
kubectl apply -f k8-manifests/collator-deployment.yml

We should do the same for Luhn Network. The only configuration differences are:

use --chain luhn instead of --chain hashed
the names of objects in k8s have prefix of hashed instead of md5 (except for the namespace, which stays hashed-network unless there is a desire to create new namespaces for luhn and md5)

Each collator should be deployed twice, with DNS entries as:

c1.hashed.network
c2.hashed.network
c1.luhn.network
c2.luhn.network

Once deployed, the RPCs and chain state will be up and available, but of course it won't build blocks until it has a parachain lease. When we get close to that time, we can inject the keys required for block signing.
In the meantime, we will continue to build and we can upgrade the nodes and our runtime on Polkadot. However, there is a transaction cost of 100 DOT per on-chain upgrade so we need to be judicious. CC @tlacloc @didiermis

@max I don't see the file: k8-manifests/collator-deployment.yml, I see: k8-manifests/collator-manifest.yml is this the one you are trying to deploy?

@sebastianmontero yes, that's right.  It is failing when creating the deployment, but that command was a typo; it was copied from a file where I was I debugging. I corrected.

@3yekn is there a way to open a ticket with google? it seems that the issue is "current unavailability of a Compute Engine resource, for example GPUs or CPUs in the requested zone" this is why the node pool is not scaling up, I tried creating a node pool in another zone, but the cluster was created with no nodes. I guess this is a temporary issue, but not sure.

If you are logged in and follow through to support and chat, it would route to an agent that will either be able to help or will escalate it to someone else. On Nov 28, 2022, at 5:15 PM, sebastianmontero @.***> wrote:
@3yekn is there a way to open a ticket with google? it seems that the issue is "current unavailability of a Compute Engine resource, for example GPUs or CPUs in the requested zone" this is why the node pool is not scaling up, I tried creating a node pool in another zone, but the cluster was created with no nodes. I guess this is a temporary issue, but not sure.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

@3yekn  Yes, I've tried it but seems I can only access billing support:

@3yekn the chain argument does not seem to be working both are being deployed as luhn, which docker file did you use to build the image?


Feed Images with Variable length

Hey guys i am trying to use the model to train on the images with variable length size.As you know this data set is a fixed length size and there is no problem with running the model but when it comes to other datasets such as IAM we get error since the are not fixed size.One of the techniques that have been mentioned is to do zero padding. Now my question is am i suppose to do the zero padding on the images itself before feeding them to the network and is there any other ways to overcome this issue of variable sizes.
Thanks

＠watsonyanghx would you please shed some light on this? :)


Third party repository detection

Description
Some plugins are using third party repositories to resolve dependencies.
Being able to detect those could be of interest for the infra team.
Idea from @lemeurherve and @dduportal

For instance:

https://github.com/jenkins-infra/helpdesk/issues/3501
https://github.com/jenkins-infra/helpdesk/issues/3434

Hi!
I would love to work on this issue

Hi everyone,
If a <url> in pom.xml does not start with https://repo.jenkins-ci.org/ui/packages, does that mean it is a third-party package?
Should this be my approach to detect a third party package?

@DanielS01ss I'm sorry, but as part of the GSoC project, it would be better if you could find another issue to work one. Those without the project GSoC 2023 are up for grab.
@Jagrutiti no really. This because you are project.url which should point to the plugin repository (or somewhere in the repository) for the plugin documentation migration. But you have project.repositories which should be a list of 1 or more elements. What we want is to detect if a plugin is using a repository which is not https://repo.jenkins-ci.org/public.
Please see https://maven.apache.org/guides/introduction/introduction-to-repositories.html and sub-documentation to have more details.
Note: please reuse what exists to parse the pom.xml, aka use a library to read the POM file.

After discussing with @lemeurherve, we need to track everything that is downloaded outside of repo.jenkins-ci.org. This also includes pluginRepositories.

To continue the work further, learning more about effective-pom and how to run maven from java could prove helpful.

@alecharp Please add GSOC label


Backspace doesn't work in active code pane

VSCode Version: 1.13.0
OS Version: Android Linux 3.4.113-Lineage-ga22c622

Steps to Reproduce:

Open a file in VS Code
Place cursor at right edge of existing text
Press backspace key
Expect characters to be removed "right to left"
No characters are removed.

The backspace key does function in the extensions search window, but not in the code editor window.
This makes it very important to write perfect code on the first attempt.

I also have the same problem, backspace key not work on editor window, and the 5 key is not input a 5 number. 5 key is instead backspace key behavior

the follow is similar problem in my raspberrypi:
code-oss armhf 1.29 with vim extension:
at begin:

press ESC will trigger the keymap setting, not change the vim status
the backspace not response in text editor.
change the follow item:
File--->Preference--->setting---->change the keyboard item which under the application,
from code to keyboard

now the text editor work like the vim


Update manifests

I noticed that you haven't been updating the manifest here in a while, I updated about 70 of them for you.

We want to keep the commit log syncing with the scoop repository, so this pull request will not be merged. We will rebase the branch based on the latest commit log of the scoop repository. for more information, see https://github.com/scoopinstaller/scoop-core/issues/1#issuecomment-459680063

Oh, OK.


Documents Page file upload error

To Reproduce
Steps to reproduce the behavior:

Go to Documents Page
Open any company for exp Ajayraj
Create new folder name test, and drag and drop image file in it.

try to upload files by browse files gives this error

possible solve


Postgres - No Expiry not working

When I use postgres storage, I tried to set 0 for no expiry... but the keys are expiring when set to 0

Fix is coming with the next release


Newline in error message breaks intake

Describe the issue:
Since #9677, some dask error messages (e.g. file not found) have included a newline.
intake forwards error messages in a way that doesn't allow newlines, and hence fails its tests: failure log.
This could be fixed in dask by removing the newline (but keeping the rest of the new wording).  I don't know if it could reasonably instead be fixed in intake: @martindurant
Environment:

Dask version: 2022.12
Python version: fails in both 3.10 and 3.11
Operating System: Ubuntu

Would be happy to fix it in intake. We can transfer the issue or perhaps close this one. Do you think you make a simple non-dask reproducer?

Thanks for the issue @rebecca-palmer and for handling things in intake @martindurant

From the logs,  it seems the ValueError due to the new line is raised in tornado more specifically, here https://github.com/tornadoweb/tornado/blob/b726f6d66133189a1cf25a695e4e1c96d70c713b/tornado/http1connection.py#L455-L457


icon problems

so having some problems with the icon feature
setting it via the defaults, or with the ->withIcon() method is not setting the icon on messages sent.
Using the v1.7.0 from packagist

Thanks, will look into it later today.
On Thu, Jun 11, 2015 at 2:56 AM, Zxurian<EMAIL_ADDRESS>wrote:

so having some problems with the icon feature
setting it via the defaults, or with the ->withIcon() method is not setting the icon on messages sent.
Using the v1.7.0 from packagist
Reply to this email directly or view it on GitHub:
https://github.com/maknz/slack/issues/24

@Zxurian I can't reproduce this, everything is working totally as expected.
With an emoji, with withIcon:
$c = new Maknz\Slack\Client('http://webhook.url')
$c->to('@regan')->withIcon(':ghost:')->send('hello from ghost');

With a URL:
$c->to('@regan')->withIcon('https://slack.com/img/icons/app-57.png')->send('hello from slack');

and via defaults:
$c = new Maknz\Slack\Client('http://webhook.url', ['icon' => ':heart_eyes:']);
$c->to('@regan')->send(':heart:');

A couple of things for you to try:

If you're using a URL, make sure the URL is valid and accessible everywhere. If Slack can't load it, it won't display it.
If you're using emoji, make sure the emoji is valid and is properly delimited by semicolons :. Try pasting what you're using as your icon into Slack to test.
If there has been no other chat in Slack since the icon change, and the same bot speaks next, the icon won't change, as the messages will all stack under the same icon/username to save space, so no new icon is requested. If you break up the chat by typing something and try again, the bot should get the new icon.


chore(formatting): Introduce npm scripts for automatic formatting using Prettier

Prettier is now a dev dependency
package.json is now sorted alphabetically so it's easier to find fields
there are two new scripts: format and format:fix, to verify and fix formatting
default indentation for all files is two spaces

I hope you like the changes, I didn't commit reformatted files on purpose until the configuration is approved. I'll format all of the files before merging.
Resolves #1342

Ok, you can take a look at .editorconfig, I've only changed line length for md and yml files to 80 characters so it's more readable on GitHub (wrapped lines in indentation-sensitive yml is annoying, for markdown it's mostly text and there is some research made by Medium that text files should be quite narrow so people can read it faster :) )

And there are conflicts somewhere

@merlinnot shall I squash it? Or can you amend some of the commits? I actually can't rebase and merge it.. Github says there are conflicts

can you amend this commit Merge branch 'master' into prettier

Amend is useful for changing commits. How would you like me to change this commit?
I can merge branch prettier to master locally without any issue.


retryAttempts exceeding maxRetriesPerRequest, goes on reconnecting indefinitely for new Redis({options_come_here})

Trying to establish new Redis connection using ioredis. Once the server goes down, the client is not ending after maxRetriesPerRequest. retryStrategy is called indefinitely.

Hey @hamvisa. The two options are not related. retryStrategy will be called every time the connect attempt is failed. You may want to return a non-number value to discard further attempts. maxRetriesPerRequest is to single command instead of the whole connection. A command will be rejected when retry attempts exceed maxRetriesPerRequest but the whole connection may still retry based on the return value of retryStrategy.


Unknown custom metadata item kind: 7

See http://apicomparer.particular.net/compare/RavenDB.Client/3.0.30155...3.5.0-rc-35169

RavenDB.Client comparison between 3.0.30155 and 3.5.0-rc-35169 has failed with the following error: Unknown custom metadata item kind: 7

Verified that this works locally. Wonder if have some stuff installed that makes this work?

Works now, the nuget update we did a while back was the fix but the deploy was broken


qemu: Update qemu to qemu 5.0.0

This is primarily being done since I want raspi3 support,
however it appears that our qemu is at least a month old

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Duplicate of https://github.com/NixOS/nixpkgs/pull/86291
And what's with the patches and foo thing?


Update EventGrid sample repo for track2

https://github.com/Azure-Samples/event-grid-python-public-consume-events

https://github.com/Azure-Samples/event-grid-python-public-consume-events/pull/4
Verified that the corresponding samples exist in trac 2

@lmazuel  tagging you on this so we can decide on how to tackle the mgmt samples

@rakshith91 Let's do a simple disclaimer in the front Readme to say "This repo is about azure-eventgrid v1, if you are using azure-eventgrid v2, please look here . This repo will be archived once the v2 will be released as stable"


Callback warning error

I receive the following error when using fit_generator with callbacks in Keras:
/opt/conda/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/callbacks.py in _call_batch_hook(self, mode, hook, batch, logs)
     93                 'Method (%s) is slow compared '
     94                 'to the batch update (%f). Check your callbacks.', hook_name,
---> 95                 delta_t_median)
     96         if hook == 'begin':
     97             self._t_enter_batch = time.time()

TypeError: integer argument expected, got float

Apparently it happens with some combination of warnings package and Keras packages, or something else. Any idea for a quick workaround that does not involve removing callbacks?

Hello,
I have the same problem at Kaggle when implementing cyclical learning rate using the Callback Class.  And it is very strange for me, because the code works only with VGG16. But with wother networks I get the message:
Epoch 1/10
2/91 [..............................] - ETA: 29:56 - loss: 0.9238 - acc: 0.4500
TypeError                                 Traceback (most recent call last)
 in ()
26                               validation_steps = len(val_list) // batch_size + 1,
27                               callbacks=[clr],
---> 28                               verbose=1)
/opt/conda/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
89                 warnings.warn('Update your ' + object_name + ' call to the ' +
90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
92         wrapper._original_function = func
93         return wrapper
/opt/conda/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
1434             use_multiprocessing=use_multiprocessing,
1435             shuffle=shuffle,
-> 1436             initial_epoch=initial_epoch)
1437
1438     @interfaces.legacy_generator_methods_support
/opt/conda/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
217                     batch_logs[l] = o
218
--> 219                 callbacks._call_batch_hook('train', 'end', batch_index, batch_logs)
220
221                 batch_index += 1
/opt/conda/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/callbacks.py in _call_batch_hook(self, mode, hook, batch, logs)
93                 'Method (%s) is slow compared '
94                 'to the batch update (%f). Check your callbacks.', hook_name,
---> 95                 delta_t_median)
96         if hook == 'begin':
97             self._t_enter_batch = time.time()
TypeError: integer argument expected, got float
Please help me to solve this problem.
Best


Missing trailing slash on proxy_pass to frontend

Your original nginx config produced by default installation process.
server {
  listen 80;
  server_name ambar-fe;
  client_max_body_size 1024m;
  location /api/ {
    proxy_pass http://webapi:8080/api/;
  }
  location / {
    proxy_pass http://frontend:80;
  }
}

The last directive should be proxy_pass http://frontend:80/;

This has been fixed with RD17/ambar-proxy@328084c9d124f2d51e8030bd994167ab8b5d742a, could you build a new docker image?

@luxflux Do you guys have some kind of chat? I can't get ambar running on my machine at all. Upload function isn't working

@luxflux will update images ASAP

@thangngoc89 please, create a new issue with description of your problem

@luxflux @thangngoc89 images updated


adding obj relationship on Utxo to Transaction on txHash=hash.

Certain applications of this repository one to fetch the Block number
of a Utxo along with the Utxo itself.
This is problematic as, to the best of your author's knowledge, there
is no way to do this except by first querying the Utxo, then using its
txHash to query the Transaction.
An unfortunate circumstance as it is, God forbid that one does this
inside of a loop of Utxos!
As far as your author can tell, a Utxo has a logical link to one and
only one transaction, so this unfortunate circumstance can be remedied
simply by making Hasura and company aware of the logical link.
Then one can fetch a the Block.number of a Transaction.hash that matches
one's Utxo.txHash.
Happy days!
Your author would be grateful if a knowledgable expert could correct his
poor understanding of this repository revealed in any mistakes made in
this PR, or else if the same expert could advise on other ways to avoid
the unfortunate circumstance described herein.

Closing in favour of the linked PR


minimize with LinearConstraint IndexError: boolean index did not match indexed array along dimension 0;

Hi, I am trying to minimize a function with method('trust-constr') and I construct a linear constraint. But there occurs this error: "IndexError: boolean index did not match indexed array along dimension 0; dimension is 1 but corresponding boolean dimension is 6"

My code is as follows:

Here is the text version:
from scipy.optimize import SR1
from numpy import np
init_guess = [0.1,0.1,0.1,0.1,0.6]
a = np.matrix(np.identity(5))
b = np.matrix([1,1,1,1,1])
c = np.concatenate((a, b), axis=0)
lb = [0,0,0,0,0,1]
ub = [1,1,1,1,1,1]
constr = scipy.optimize.LinearConstraint(c, lb, ub, keep_feasible=True)

max_sharpe = scipy.optimize.minimize(neg_sharpe, init_guess, args =(COV, mu, rf),
                                     constraints=[constr], jac="2-point", hess=SR1(),
                                     method='trust-constr',
                                     options={'verbose': 1, 'disp': True, 'maxiter': 10000})

I wonder if anyone can help me with this. Thanks a lot!

Hi @yl-hz . Could you please post your code as text and not as an image?

Hi @yl-hz . Could you please post your code as text and not as an image? Also try with your bounds being 2D arrays instead

Sure! I have updated my question! Thanks!

Hi @yl-hz . Could you please post your code as text and not as an image? Also try with your bounds being 2D arrays instead

Sure! I have updated my question! Thanks!

I tried to convert all c, lb and ub into np.array and it works! Thank you so much!!

@yl-hz Glad that it works. We can close this then.
@mdhaber @Kai-Striega when you have time could you please check whether we should update the docs or the entrance code such that lists already works? The example is not reproducible but maybe it is a genuine docs issue.

@antonior92 Do you have a moment to check this? I think this is more in your domain, since it is about using a LinearConstraint object for use with 'trust-constr'.

Hi @ilayn, the problem here is not the use of lists (which works), but rather the use of np.matrix.
The current expected type for A is A : {array_like, sparse matrix}, shape (m, n). Do you think we should change it?

Since we are trying to get rid of numpy.matrix eventually, I'd vote for not mentioning it. Do you think an explicit asarray() needed to conform with array_like acceptance spec?

We already do some treatment of the matrix A:
class LinearVectorFunction(object):
    """Linear vector function and its derivatives.
    Defines a linear function F = A x, where x is n-dimensional vector and
    A is m-by-n matrix. The Jacobian is constant and equals to A. The Hessian
    is identically zero and it is returned as a csr matrix.
    """
    def __init__(self, A, x0, sparse_jacobian):
        if sparse_jacobian or sparse_jacobian is None and sps.issparse(A):
            self.J = sps.csr_matrix(A)
            self.sparse_jacobian = True
        elif sps.issparse(A):
            self.J = A.toarray()
            self.sparse_jacobian = False
        else:
            self.J = np.atleast_2d(A)
            self.sparse_jacobian = False

        self.m, self.n = self.J.shape

        self.x = np.atleast_1d(x0).astype(float)
        self.f = self.J.dot(self.x)
        self.f_updated = True

        self.v = np.zeros(self.m, dtype=float)
        self.H = sps.csr_matrix((self.n, self.n))

I sincerely thing it is enought to comply with the specification. What do you think?

Same here, I also think it is sufficient. I was just a bit surprised that np.matrix doesn't get normalized after atleast_2d.

I was just a bit surprised that np.matrix doesn't get normalized after atleast_2d.

Me too :)


Backport of xyna-zeta #224 and xyna-process-modeller #137

https://github.com/Xyna-Factory/xyna-process-modeller/pull/137
https://github.com/Xyna-Factory/xyna-zeta/pull/224

Closed, wrong repo.


ci(pipeline.yml): #275 add cog-bump to calculate the next bump version

this PR is not needed anymore.


[admin[ Updating contributor guide

This PR is based on the updates in https://github.com/OfficeDev/office-js-docs-reference/pull/1153. If leaves the existing verbage explaining sample contributions, since the audience is slightly different.

Docs Build status updates of commit 2739390:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Contributing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 8e5fda3:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Contributing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit ced5ce3:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Contributing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 65a7c1d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Contributing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 0e9cc1a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Contributing.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Fix compile app with graph which use string as original id.

This PR fixed string not recognized in some frame files cause the std prefix is not there.
This PR also add a module level doc string for graphscope, which fixes #104 .
But when digging deep in whether all apps could run on graph which use string as original id, it seems that most apps will error out when quering. The reason includes 1. AutoWorkers cannot send string messages. 2. Some apps are not designed with support string in mind, so weird error happens, etc. These require a more careful case by case study.

Codecov Report

Merging #120 (a1f33e3) into main (512a5b8) will decrease coverage by 13.14%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main     #120       +/-   ##
===========================================
- Coverage   78.30%   65.16%   -13.15%
===========================================
  Files          50       49        -1
  Lines        4601     4403      -198
===========================================
- Hits         3603     2869      -734
- Misses        998     1534      +536

Impacted Files
Coverage Δ

python/graphscope/framework/dag_utils.py
63.52% <ø> (ø)

python/graphscope/interactive/__init__.py
0.00% <0.00%> (-100.00%)
:arrow_down:

python/graphscope/interactive/query.py
0.00% <0.00%> (-94.65%)
:arrow_down:

python/graphscope/deploy/kubernetes/utils.py
12.69% <0.00%> (-61.43%)
:arrow_down:

python/graphscope/deploy/kubernetes/cluster.py
22.34% <0.00%> (-52.84%)
:arrow_down:

...n/graphscope/deploy/kubernetes/resource_builder.py
28.14% <0.00%> (-32.52%)
:arrow_down:

python/graphscope/analytical/app/triangles.py
80.00% <0.00%> (-20.00%)
:arrow_down:

python/graphscope/client/rpc.py
65.32% <0.00%> (-18.51%)
:arrow_down:

python/graphscope/client/utils.py
79.16% <0.00%> (-16.92%)
:arrow_down:

python/graphscope/framework/graph.py
82.67% <0.00%> (-9.37%)
:arrow_down:

... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 512a5b8...a1f33e3. Read the comment docs.

Could you please DON'T mix some unrelated doc changes into a bugfix PR?


Bytecode VM

Sauros is walking the lists that it parses. Creating a virtual machine that reads a byte code could potentially be much faster. This would entail designing and documenting a byte code, creating something to fully walk a list (down every possible execution branch) and emit that byte code, and a VM that executes said byte code. This VM and byte code would need some way of referencing locations in the file (hopefully) in the event of things like type errors so we can direct the programmer to the source of the issue.

No longer planned in the short-to-long term


segger-jlink: 766 -> 792b

Description of changes
Version bump.
Tested for flashing of a local project, which works.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 252137 run on x86_64-linux 1

I updated it to 792b, two versions newer than 792.
I thought b denoted a beta build, but it doesn't, as previous versions seem to go up to n.

I don't know why the review does not build the package... :thinking:

I think this is superseded by #255185


[WIP] Fully port InverseCancellation to Rust

Summary
This commit builds off of #12959 and the other data model in Rust
infrastructure and migrates the InverseCancellation pass to
operate fully in Rust. The full path of the transpiler pass now never
leaves Rust until it has finished modifying the DAGCircuit. There is
still some python interaction necessary to handle parts of the data
model that are still in Python, mainly for handling parameter
expressions. But otherwise the entirety of the pass
operates in rust now.
This is just a first pass at the migration here, it moves the pass to
use loops in rust. The next steps here are to look at operating
the pass in parallel. There is no data dependency between the
optimizations being done for different inverse gates/pairs so we should
be able to the throughput of the pass by leveraging multithreading to
handle each inverse option in parallel. This commit does not attempt
this though, because of the Python dependency and also the data
structures around gates and the dag aren't really setup for
multithreading yet and there likely will need to be some work to
support that.
Details and comments
This PR is based on top of https://github.com/Qiskit/qiskit/pull/12959 and as such github shows the entire contents of https://github.com/Qiskit/qiskit/pull/12959 in addition to the contents of this PR. To see the contents of this PR you can look at HEAD on this branch, or just look at:
https://github.com/Qiskit/qiskit/pull/13013/commits/76146893dba67a43b208c1b82de686e9940d6809
Fixes #12271
Part of #12208
TODO:

[ ] Rebase after #12959 merges
[ ] Benchmark and tune the implementation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

5256 of 6222 (84.47%)  changed or added relevant lines in 45 files are covered.
66 unchanged lines in 11 files lost coverage.
Overall coverage decreased (-0.5%) to 89.124%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

crates/circuit/src/interner.rs
10
11
90.91%

crates/circuit/src/circuit_data.rs
21
23
91.3%

crates/accelerate/src/nlayout.rs
3
6
50.0%

crates/accelerate/src/inverse_cancellation.rs
138
144
95.83%

crates/circuit/src/circuit_instruction.rs
3
9
33.33%

crates/circuit/src/operations.rs
10
17
58.82%

crates/circuit/src/packed_instruction.rs
65
72
90.28%

qiskit/visualization/dag_visualization.py
5
20
25.0%

crates/accelerate/src/euler_one_qubit_decomposer.rs
151
167
90.42%

crates/circuit/src/dag_node.rs
167
193
86.53%

Files with Coverage Reduction
New Missed Lines
%

crates/circuit/src/interner.rs
1
74.07%

qiskit/circuit/instruction.py
1
95.64%

qiskit/dagcircuit/dagnode.py
1
85.37%

qiskit/transpiler/passes/optimization/optimize_1q_decomposition.py
2
94.23%

crates/circuit/src/circuit_instruction.rs
4
88.5%

qiskit/dagcircuit/dagdependency_v2.py
5
67.62%

crates/qasm2/src/lex.rs
5
92.48%

crates/circuit/src/dag_node.rs
7
83.61%

crates/accelerate/src/euler_one_qubit_decomposer.rs
8
91.49%

crates/circuit/src/operations.rs
14
88.19%

Totals

Change from base Build<PHONE_NUMBER>8:
-0.5%

Covered Lines:
71659

Relevant Lines:
80404

💛  - Coveralls