Memory leak of pulsar-function-go library

Describe the bug
It seems that there is a memory leak in the pulsar-function-go library.
I implemented a simple pulsar function worker that just write logs using pulsar-function-go/logutil for sending logs to log topic. I tried to long-term test by sending request messages consecutively to the input topic to check the feasibility.
At the first time, I set --log-topic for the function worker, but I faced a ProducerQueueIsFull error after a few seconds later.
After that I didn't set --log-topic option to find out the reason. From the second test, there was no more ProducerQueueIsFull error, but the memory of the pulsar function worker process had grown indefinitely.
I used pprof to pinpoint the root cause of the problem. Please refer to the result below.
$ go tool pprof -top http://localhost:6060/debug/pprof/heap
Fetching profile over HTTP from http://localhost:6060/debug/pprof/heap
File: simple-worker
Build ID: bbb7d25540b7a482661b05e4c0da0a5ba2ef7bae
Type: inuse_space
Time: Nov 8, 2019 at 3:44pm (KST)
Showing nodes accounting for 12.51MB, 100% of 12.51MB total
      flat  flat%   sum%        cum   cum%
   12.51MB   100%   100%    12.51MB   100%  github.com/sirupsen/logrus.(*Entry).String
         0     0%   100%    12.51MB   100%  github.com/apache/pulsar/pulsar-function-go/logutil.(*contextHook).Fire
         0     0%   100%        1MB  7.99%  github.com/apache/pulsar/pulsar-function-go/logutil.Error
         0     0%   100%    11.51MB 92.01%  github.com/apache/pulsar/pulsar-function-go/logutil.Infof
         0     0%   100%        1MB  7.99%  github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).addLogTopicHandler
         0     0%   100%    11.51MB 92.01%  github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).handlerMsg
         0     0%   100%    12.51MB   100%  github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).startFunction
         0     0%   100%    12.51MB   100%  github.com/apache/pulsar/pulsar-function-go/pf.Start
         0     0%   100%    11.51MB 92.01%  github.com/apache/pulsar/pulsar-function-go/pf.newFunction.func1
         0     0%   100%    11.51MB 92.01%  github.com/apache/pulsar/pulsar-function-go/pf.pulsarFunction.process
         0     0%   100%    12.51MB   100%  github.com/sirupsen/logrus.(*Entry).Log
         0     0%   100%    11.51MB 92.01%  github.com/sirupsen/logrus.(*Entry).Logf

...
...

As you could see, a string object for logging had grown during the test.
It seems that ProducerQueueIsFull error with --log-topic option, and the memory leak without --log-topic option is caused by the same reason. So I modified pulsar-go-function codes to fix it. Fortunately, It seems has been fixed now. So I will send a pull request for that.
To Reproduce
Steps to reproduce the behavior:

Prepare a pulsar cluster with standalone mode.
Prepare a simple pulsar function worker. Refer to the codes below.

$ cat simple-worker.go
package main

import (
        "context"
        "fmt"
        "net/http"
        _ "net/http/pprof"

        log "github.com/apache/pulsar/pulsar-function-go/logutil"
        "github.com/apache/pulsar/pulsar-function-go/pf"
)

func main() {
        // go routine for pprof
        go func() {
                fmt.Println("%+v", http.ListenAndServe("localhost:6060", nil))
        }()

        pf.Start(testFunction)
}

func testFunction(ctx context.Context) {
        for i := 0; i < 10; i++ {
                log.Infof("This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function. This is test of pulsar function.\n")
        }
}

Prepare a conf.yaml file for the function worker.

---
pulsarServiceURL: "pulsar://sunkwan-devpc:6650"
instanceID: 0
funcID: "ed5f48f4-8e76-4d30-8b88-6eea1ff3cbf3"
funcVersion: "12f361a0-67e5-4404-8177-8c328a33e3db"
maxBufTuples: 1024
port: 0
clusterName: "standalone"
killAfterIdleMs: 0
tenant: "public"
nameSpace: "default"
name: "func-concat"
className: ''
#logTopic: 'persistent://public/default/func-concat-logs'
logTopic: ''
processingGuarantees: 0
secretsMap: ''
runtime: 0
autoAck: true
parallelism: 3
subscriptionType: 0
timeoutMs: 0
subscriptionName: ''
cleanupSubscription: true
sourceSpecsTopic: "persistent://public/default/func-concat"
sourceSchemaType: ''
receiverQueueSize: 1
#sinkSpecsTopic: "persistent://public/default/func-concat-output"
#sinkSpecsTopic: ""
#sinkSchemaType: ''
#cpu: 1
#ram:<PHONE_NUMBER>
#disk:<PHONE_NUMBER>0
#maxMessageRetries: 0
#deadLetterTopic: ''
#regexPatternSubscription: false

Execute the simple pulsar function worker by command line below.

./simple-worker --instance-conf-path ./conf.yaml

Send messages to the input topic consecutively.
Check the memory usage and heap status using pprof.

$ go tool pprof -top http://localhost:6060/debug/pprof/heap
Fetching profile over HTTP from http://localhost:6060/debug/pprof/heap
Saved profile in /root/pprof/pprof.simple-worker.alloc_objects.alloc_space.inuse_objects.inuse_space.120.pb.gz
File: simple-worker
Build ID: bbb7d25540b7a482661b05e4c0da0a5ba2ef7bae
Type: inuse_space
Time: Nov 8, 2019 at 3:48pm (KST)
Showing nodes accounting for 30080.51kB, 100% of 30080.51kB total
      flat  flat%   sum%        cum   cum%
29211.22kB 97.11% 97.11% 29211.22kB 97.11%  github.com/sirupsen/logrus.(*Entry).String
  869.29kB  2.89%   100% 30080.51kB   100%  github.com/apache/pulsar/pulsar-function-go/logutil.(*contextHook).Fire
         0     0%   100%  1536.21kB  5.11%  github.com/apache/pulsar/pulsar-function-go/logutil.Error
         0     0%   100% 28544.30kB 94.89%  github.com/apache/pulsar/pulsar-function-go/logutil.Infof
         0     0%   100%  1536.21kB  5.11%  github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).addLogTopicHandler
         0     0%   100% 28544.30kB 94.89%  github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).handlerMsg
....

Expected behavior

ProducerQueueIsFull error shouldn't be occurred.
Memory usage of pulsar worker process shouldn't be grown indefinitely.

Screenshots
N/A
Desktop (please complete the following information):

OS: Ubuntu 18.04

Additional context
N/A

Is there any update for merging?


Markers

Eliminated the marker images and implemented a modified version of Noodlekit's marker. The gradient and tweaks make it appear nominally the same as the old bitmap.
In order to keep the current delegate API the same, I'm considering adding another delegate. Current delegate (only) results in current behavior and appearance (unless the color property is set). Another delegate could be called to get the correct color for the line.
It shouldn't be too hard to add another delegate to provide an image for the line, but then the choice is we're deluged in different types of delegate methods, or else the very simple current implementation becomes complex for the implementing application. I'm satisfied with being able to change the color, but I don't speak for everyone.

Closing this -- something when wrong when I merged with your latest changes.


Add ISupportSampling and implement it on Telemetry types that support it.

@SergeyKanzhelev, @vitalyf007, @JakubOleksy please have a look.

:shipit: with the change of condition in serialization

Looks good to me.

:shipit:


GCM Android Push notification issue

Thanks for such nice tutorial. However I am using same App from Screen shot to register into GCM, but its not allowing to do so.
Could you please let me know which is correct app I need to installed from my Android Device ?
Why following not working
curl --header "Authorization: key=AXXXXXXXXXXXXXXXXXXXXXXXXX" --header Content-Type:"application/json" https://android.googleapis.com/gcm/send  -d "{\"registration_ids\":[\"XXXXXXXXXXX\"],\"data\":{\"name\":\"Hello, Good Morning\"}}"
Its giving me following error. Could you please help me ASAP?
{"multicast_id":XXXXXXXXX,"success":0,"failure":1,"canonical_ids":0,"results":[{"error":"InvalidRegistration"}]}
Also, please let me know how to get RegID or GCMRegId.txt? Is there any simulator or client I can used it ? Please help / guide / support.

@pashtika From https://developers.google.com/cloud-messaging/server-ref#error-codes InvalidRegistration means that the token you are sending to the GCM server is invalid.
"Check the format of the registration token you pass to the server. Make sure it matches the registration token the client app receives from registering with GCM. Do not truncate or add additional characters."
You need a valid registration token that you got from GCM when you registered on the device.
"Registration has to be done by the client in order for it to receive a registration token. The client must communicate this token back to its server. The server will then use the token to send messages to the client"
As the Getting Started section on the README tells you: "To use the app you will need a sender ID to be able to register and test client side APIs, and an API key to be able to test server side APIs. For more information, see Implementing GCM Client on Android."
Basically, you need the google-services.json configuration file inside your app/ directory (see above link) and the API_KEY that you obtain from the Google Developers Console (See: https://www.youtube.com/watch?v=Vd8bIkE29U4)
In order for you to test the demo app provided in this repository, you need to obtain the above two things. Using the default sender id inside the demo app you can obtain a registration token, but you won't be able to send messages to that token because you don't have the API key of the project associated with the SENDER ID (which is the PROJECT ID defined in Google Developers Console), only Google has access to it.
If you try to send a downstream message, you will get Error 401 Unauthorized, if you try to add your own API Key but use the default SENDER ID you will get MismatchSenderId error.
If you try to send an upstream message it'll work but you won't be able to see it on the server because you don't control it.

Thank you. I think its worth to used Android studio (rather than STS/Eclipse) for importing project. Can we send images, sound and emoji in notifications? If yes, please share code snippet for reference.

@pashtika The answer is YES to all of your questions.

Thank you, but it will be more helpful for all your followers if you provide java code snippet for send images, sound and emoji in notifications

@pashtika
Sending images : You can send the image url in the payload, get it and download it, to create the notification.
Emoji's should follow the unicode format, and can be part of the text itself.
If you want to test pushes you can use http://gcmsender-1062.appspot.com/ interface. Something I use for my own code. Alternately you can use postman or curl or any rest client.
So you can send the following json in your data section or you can put it in one key and get everything.
{
    "title": "Hello, GoodMorning",
    "msg": "Your notification content",
    "img": "https://somedomain.com/someimage.jpg"
}


Question about the fetch polyfill usage note

The note about fetch says:

Uses FileReader and Blob if they exist, and depends on FormData

But I'm not clear on what this means for me as a user. The compatibility table says the fetch polyfill works in all IE versions, but caniuse says FormData is only available in IE10+. So am I supposed to polyfill FormData myself for IE<10, and if so, why doesn't polyfill.io just do this for me?

Based on https://github.com/Financial-Times/polyfill-service/pull/246#issuecomment-69977678 it looks like that note is incorrect.  I'll change it.

Even after the above PR, still a little confused if there's a workaround built-in for FormData for older versions that don't have it natively. Could you please clarify?

Unfortunately we don't have a FormData polyfill, we will gladly accept a pull-request if you know of one or are willing to create one yourself. Here are our guidelines for authoring a polyfill. If you get stuck at any step, feel free to create an issue and we will aim to help :-)

Gotcha, appreciate the response. I did come across https://github.com/moxiecode/moxie/blob/master/src/javascript/xhr/FormData.js which appears to be a fairly solid solution, but don't have my head wrapped around the internals of this lib for contributing just yet. Have you heard of this one before?


Suggested changes

Hi Angristan,
Your guide works spectacularly! I have used it many times so far. Thanks for it.
But, as far as I understood, I believe you should update it now. Arch changed the base packages and there's a need to add packages to pacstrap. I believe your pacstrap instruction should now be updated to something like:
# pacstrap /mnt base base-devel linux linux-firmware
Arch doesn't suggest installing base-devel but without it many things like the sudo command just don't work, so having it installed doesn't hurt! Additionally, I think that a text editor like nano must also be installed as an extra.
One last point regards this part, where you have:
localectl set-x11-keymap fr-latin9
I believe it should instead be, just:
localectl set-x11-keymap fr

Thank you!


Meta issue for https://github.com/flutter/flutter/issues/15277

This is here as a placeholder for https://github.com/flutter/flutter/issues/15277

landed https://github.com/flutter/flutter/pull/15376 should fix dev/tools/vitool tests in preview-dart-2


Checkbox checked prop does not persist if conditional is given

Bug report
I'm currently passing a conditional to the checked prop for the Checkbox component as such for example
<Checkbox checked={a.length === b.length}
a and b have different lengths at the beginning, but when a does have the same length as b, this error shows up

I'm also able to toggle the checkbox despite the conditional returning a boolean value
e.g a = [1, 2, 3] and b=[4, 5, 6], the checkbox will be checked correctly, but i can still click on the checkbox to uncheck it. It should remain checked due to the conditional

Oh a way to repro is just <Checkbox checked={false} /> it'll not be checked, but if you click the checkbox, it can get checked

Oh a way to repro is just <Checkbox checked={false} /> it'll not be checked, but if you click the checkbox, it can get checked

:tada: This issue has been resolved in version 0.1.15 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 0.1.15 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Analysis Server: implement support for .packages

Tracking bug. Parent/meta issue is https://github.com/dart-lang/sdk/issues/23372

As of https://github.com/dart-lang/sdk/commit/818229cbae1004f560428f843f7ebe531cf10558 support can be enabled via an option specified in a .dart_analysis_server.yaml file in your homedir. Specifically:
ContextManagerImpl.ENABLE_PACKAGESPEC_SUPPORT: true

@stereotype441 what do you think, can we close this? How do you feel about this support?

We are as complete as we're going to get for 1.12.  All the major use cases we're aware of are addressed.
Some rewrite for 1.13 is planned that should tighten up the feature even more, and reduce our risk of unknown bugs, but I don't see any reason to hold up 1.12 waiting for that.

Thanks for the update!


Which Hosting you are using ?

Can you please specify which hosting you are using ?

Can you please specify which hosting you are using ?

Im using the Firebase as Hosting  and for API im using Github Pages


Refactoring federal senate dataset

This is in progress, I just created it earlier to have earlier feedbacks.
What is the purpose of this Pull Request?
The purpose of this PR is make some improvements on tests and code of federeal_senate_dataset.
What was done to achieve this purpose?
I created a new kind of test called journey test to test the real workflow even taking more time. Would like to receive some feedbacks to check if it's good or not for the project.
At the first time that I executed it, took more than 8 minutes now it takes almost 5:30 minutes. Se output below:
(.toolbox-env) ➜  fork-toolbox git:(master) ✗ python -m unittest tests/journey/test_federal_senate_dataset.py
.
----------------------------------------------------------------------
Ran 1 test in 338.959s

OK
(.toolbox-env) ➜  fork-toolbox git:(master) ✗

I also added a mocked unit test to improve the feedback cycle.
How to test if it really works?
To run the journey test you can use the code below:
Do not forget to export env var RUN_INTEGRATION_TESTS with value 1 to do not have this test skipped
python -m unittest tests/journey/test_federal_senate_dataset.py

To run the unit test use this:
python -m unittest tests/test_federal_senate_dataset.py

Who can help reviewing it?
@anaschwendler is the person with more knowledge about this code, her feedbacks will be very helpful. Other people that can help are @Irio @jtemporal @cuducos. If you are not one of them feel free to provide me some feedbacks.
TODO

Add unit test for translate
Add unit test for clean
Improve method names

Nice To Have

Refactor fetch, translate, clean to fetch, translate, clean, convert, merge
Reflect this changes in chamber_of_deputies

This is in progress

WIP label added, we can take it out whenever just ping us here =)

Hi @lipemorais !
Thank you very much for this refactor, I'll be looking for that closely :)
In my opinion the only thing we have to keep, is the way that things works and some structures to keep everything working on the other places: rosie and serenata-de-amor.
So we need to keep the main methods: fetch(), translate() and clean() with its features. Everything besides that can be changed!
:tada:

So we need to keep the main methods: fetch(), translate() and clean() with its features. Everything besides that can be changed!

As I've said elsewhere this API is terrible. translate is translating labels and changing the file extensions, clean is cleaning data and merging things. I definitively think this is a terrible API. It's confusing, it's not semantically meaningful, it's hard to test and hard to maintain.
I think TDD can help us to restructure this API, working with stream of data, reducing temporary intermediary files (like federal_senate-2008.xz), so we fetch original files, do all treatment needed and finally write the results. If memory is an issue this an awesome opportunity to bring Dask in — and get another leverage: to risk get rid of Rosie's memory issues.
@anaschwendler can you list exactly what we're gonna break if we change the input and output of these methods?

So, @cuducos
Hm, it is ok to refactor all the API, but we may need some time to reflact it in the other repositories, I can help with that and we need to do it very carefully.

@anaschwendler can you list exactly what we're gonna break if we change the input and output of these methods?

For the Federal Senate we can work on it, because it is not on production, but for chamber_of_deputies it changes all its structure.

and get another leverage: to risk get rid of Rosie's memory issues.

For me it looks perfect, and we can work together on it.

Yay! Next week I can start working on that. If anyone is willing to study how embedded are those methods in the other repos architecture, that would be awesome. Specially translate and clean — fetch is good so far IMHO.

Changes Unknown when pulling ae7ff4ba299157e4aa9aa1f617268bf75a7ba214 on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling 60800ad9c0560a1b70288268ba55940deee4d987 on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling fa881825285143e27e74ed91172f0365afa1f925 on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling c618acafd7b682098d2800272acafc727ea0d87d on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling 1a89abd4c4067c7ff9c2690b80b5f39911c0da35 on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling f182b3e15a311ca249843d663a9bbb6edea3a394 on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling a6e5b0797c1cd59abed063a96886d7f9ab0005f4 on lipemorais:master into ** on datasciencebr:master**.

Changes Unknown when pulling c8f8a51e389decae3bea0f8a298719607181668d on lipemorais:master into ** on datasciencebr:master**.

@lipemorais In light of the last commits I wonder if wouldn't be wiser to generate the .xz from the .csv instead of having to commit them as fixtures?

@jtemporal it would be good but make the coupling between the tests biggers, something like clean will just work after translate has runned. This would let the tests more slow and not so unit as it should be.
In order to have less fixtures I'm trying to make the tests uses less fixtures as possible, I will commit it soon.

Changes Unknown when pulling e4cac56457daa9253cb7e3e828229852dd98afc5 on lipemorais:master into ** on datasciencebr:master**.

@anaschwendler @jtemporal @cuducos
I need some help to fix to understand why is this test failing in Travis CI. :(
This is my last test for this PR.
The next step are more nice to haves

Changes Unknown when pulling ba106a9844f5b79251ee76074cc76bf4c3699859 on lipemorais:master into ** on datasciencebr:master**.

No such file or directory: 'tests/fixtures/csv/federal-senate-2008.xz'

I think it fails because when you instantiate something like FederalSenateDataset('tests/fixtures/csv/', …) you won't be able to access any .xz (because they're in tests/fixtures/xz/).

@cuducos Actually when you call translate it generates the file with .xz extension. If you run it locally you will see it.

@cuducos Actually when you call translate it generates the file with .xz extension. If you run it locally you will see it.

I know. But does it generates the .xz in which folder: tests/fixtures/csv/ or tests/fixtures/xz/? That's my point.

It generates .xz file in csv folder: tests/fixtures/csv/ .

It generates .xz file in csv folder: tests/fixtures/csv/ .

Hum… confusing, but I got it. I have the impression we could abuse of mocks to avoid that mess in terms of file system. Unit tests to test small atomic stuff, and one integration test that would do everything form scratch…

Changes Unknown when pulling a108c807abb54fb1575482bdf4501acb1a9fcee8 on lipemorais:master into ** on datasciencebr:master**.

Hum… confusing, but I got it.

Yeah, it's confusing. For the first step of this refactoring I trying to isolate network tests, a next step would be do the same with file handling tests.
I'm worried about works in my machine problem, I do not find a simple way to reproduce what is happening in Travis CI. It will be necessary soon or later, would be good be able to reproduce it.
I do like the idea of use mocks, it  will make the test better. Could you a little more about it?

Could you a little more about it?

Some examples:

fetch: we don't need to actually run urlretrieve, we assume urlretrieve just works (Guido must have tested it hahaha) and must assert it was called x times with the proper arguments — so we mocked it and check for these calls, not for files written in the file system
translate: we test __translate_file with a single unit test and them in the translate method what is relevant is to know if self.__translate_file was called x times with the proper argument
__translate_file: we mock read_csv setting a pandas DataFrame we build in tests (in memory, not as a file) as a return_value and check if its keys/headers are actually translated
Etc…

This way we don't need fixtures at all for unit tests. And for integration tests we actually download data and check if the journey succeeds from the scratch.
Does that make sense?

Also minor comment on paths: Try to use the os.path.join to build the paths strings and avoid breaking things on windows ;)

Changes Unknown when pulling 63292b8188cf318eb603a1b2b3d8cf09502d3515 on lipemorais:master into ** on datasciencebr:master**.

Does that make sense?
@cuducos I do agree with everything that you just said.

fetch: we don't need to actually run urlretrieve, we assume urlretrieve just works (Guido must have tested it hahaha) and must assert it was called x times with the proper arguments — so we mocked it and check for these calls, not for files written in the file system

Actually this is done in tests/test_federal_senate_dataset.py:18

translate: we test __translate_file with a single unit test and them in the translate method what is relevant is to know if self.__translate_file was called x times with the proper argument

This is a great idea but now that translate test is working I think that this can wait a little.

__translate_file: we mock read_csv setting a pandas DataFrame we build in tests (in memory, not as a file) as a return_value and check if its keys/headers are actually translated

Agree with it. We can create a issue just refactor translate. What do you think?

This way we don't need fixtures at all for unit tests. And for integration tests we actually download data and check if the journey succeeds from the scratch.

With less fixtures we will have tests running even faster but since at the beginning was necessary 8:30 minutes and now we can run it in 6:32 seconds. I would like to merge this PR and let more improvements for a next one.
@jtemporal Thank you for your feedback. Where could I apply os.path.join in this PR to keep this code Windows friendly?
To finish the mystery. Travis was right, I made a mistake during a refactor when I changed a variable name. Here is the commit that makes it work again.

Coverage increased (+0.8%) to 25.818% when pulling e3b5e06519ceaeca26095f5a05d3f4d3e7d77290 on lipemorais:master into ae36f97f40c90588e4ea160ed004e5032f7bcde0 on datasciencebr:master.

If you run tests on windows the ones that have hardcoded path will fail =)

Where could I apply os.path.join in this PR to keep this code Windows friendly?

so, wherever there is a path ;)

[ ] L43
[ ] L55
[ ] L74
[ ] L86
[ ] L92

os.path.join FTW ; )

@jtemporal Thank you for feedbacks, I made the changes that you suggested and would like to see if now it's ok for you.

@lipemorais @cuducos @jtemporal hello everybody! I'm totally new here! @lipemorais can you point  me out exactly where is the journey test that takes around 6/7 minutes?

Hey, @gustavo-freitas ! I'm glad to have you here.
This is the journey test
To make it run use:
Obs.: Python3 is expected and may some dependencies needed to run this test.
python -m unittest tests/journey/test_federal_senate_dataset.py

@jtemporal I made the fixes that you suggested and opened a issue to make it work for the whole project too. Thank you for your feedbacks.

Ok… for me just one last command: we don't have fixtures in the test/fixtures directory. We actually have full datasets there. Due to the strategy adopted in this PR I understand we need these files there. But this is bizarre:
$ du -h -d 1
 15M	./.git
 44K	./docs
280K	./htmlcov
256K	./serenata_toolbox
 20K	./serenata_toolbox.egg-info
 51M	./tests
 67M	.

What I mean is: the whole thing excluding .git/ has 52Mb. 98% of the repo is in the tests/fixtures/ directory:
$ du -h -d 1 tests/fixtures
 47M	tests/fixtures/csv
3.9M	tests/fixtures/xz
 51M	tests/fixtures

IMHO we can be more concise when it comes to fixtures. We can:

edit fixtures to have 2 lines each, for example (or more only if we have a plausible reason to add more)
edit test routines to need only data from 2 years (e.g. 2008 and 2009)

This seams enough for testing purposes… what do you think @jtemporal?

Sorry… that wasn't my last comment. The test suit is leaving a residual file behind, we need to fix that before merging:
~/serenata-toolbox(lipemorais-master) » git status
On branch lipemorais-master
nothing to commit, working tree clean

~/serenata-toolbox(lipemorais-master) » python -m unittest discover tests
s..........................Could not find config.ini file.
You need Amazon section in it to interact with S3
(Check config.ini.example if you need a reference.)
.You need an Amazon section in config.ini to interact with S3 (Check config.ini.example if you need a reference.)
.Could not find config.ini file.
You need Amazon section in it to interact with S3
(Check config.ini.example if you need a reference.)
.Could not find config.ini file.
You need Amazon section in it to interact with S3
(Check config.ini.example if you need a reference.)
.....It looks like you have an old version of the config.ini file. We do not need anymore the service (s3) appended to the region (sa-east-1). Please update your config.ini replacing regions like `s3-sa-east-1` by `sa-east-1`.
.....While translating Seranata Toolbox not found file: tests/fixtures/csv/federal-senate-2007.csv
File b'tests/fixtures/csv/federal-senate-2007.csv' does not exist
..While fetching Seranata Toolbox not found file: /var/folders/lr/gd0yq9m90rv_gz682gqt7ylc0000gn/T/federal-senate-2007.csv
HTTP Error 404: Not Found
..Loading receipts.csv…
Dropping rows without document_value or reimbursement_number…
Grouping dataset by applicant_id, document_id and year…
Gathering all reimbursement numbers together…
Summing all net values together…
Summing all reimbursement values together…
Generating the new dataset…
.
----------------------------------------------------------------------
Ran 45 tests in 5.566s

OK (skipped=1)

~/serenata-toolbox(lipemorais-master*) » git status
On branch lipemorais-master
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	tests/fixtures/csv/federal-senate-2008.xz

nothing added to commit but untracked files present (use "git add" to track)

~/serenata-toolbox(lipemorais-master*) »

In sum:

I addressed the fixtures size (and some minor details) in a another branch based on this PR — feel free to take the commits if they help
What's really missing is to address the residual files… this is still an issue, ok?

In the aforementioned branch I might have made things worse. Two residual changes after testing:
On branch cuducos-lipedemorais-master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   tests/fixtures/xz/federal-senate-reimbursements.xz

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	tests/fixtures/csv/federal-senate-2008.xz

no changes added to commit (use "git add" and/or "git commit -a")

In 704f675 you removed some fixtures… are you shrinking the remaining ones to a couple of lines? The ones from my branch have 2 reimbursements each, pretty compact — feel free to grab them ; )

edit fixtures to have 2 lines each, for example (or more only if we have a plausible reason to add more)

I agree with @cuducos on this. If you want to test for all of the years reducing the fixtures to the minimum amount of lines in each would be the way to go IMHO

@jtemporal @cuducos I completely agree, I removed some datasets that are not necessary anymore but there is space for improvement. I'm planing to fix this soon.

What's really missing is to address the residual files… this is still an issue, ok?

OK. I working on it.
I addressed the name convention here 0dfb560cae880b836d15b926712e1640747f4390

In 704f675 you removed some fixtures… are you shrinking the remaining ones to a couple of lines? The ones from my branch have 2 reimbursements each, pretty compact — feel free to grab them ; )

I will grab just the idea, because the encoding is different. ;)

@cuducos and @jtemporal I tunned the fixtures as much as I could to make it very fast eliminating generated files after somo tests that needed it.
Do you see any space for improvements?

Hell yeah! Many thanks @lipemorais for that improvements and @jtemporal and @anaschwendler for the support and code reviews  🎉

I believe now it is only missing a version bump as @anaschwendler kindly reminded me earlier =)

Done.


How to utilize temperature scaling in binary classification problem with Neural Nets?

Any hints? Or should I just use the traditional method like Platt

@ydzhang12345 apologies for not responding to this issue earlier! It must've slipped in my notifications.
You can apply temperature scaling as is to binary classification problems. In this case, it is almost exactly identical to Platt scaling. Just apply the temperature to the final output value of the network.

Thank you. I did try that. Turns out using Platt will be better. Thank you!

@ydzhang12345 how did you utilize the temperature scaling ? sigmoid(logits / T) or softmax(logits / T) ? what op does your original network to use to compte prob? sigmoid or softmax ?


Add build option to build a static lib

Would you consider a PR to add an option to build statically (libx86simdsortcpp.a) ?
https://github.com/intel/x86-simd-sort/blob/87486d17a7d916f2992800b0219e28d0af29e765/meson.build#L37

@WilliamTambellini sure. Could you make it an optional meson argument that defaults to shared?


redux of mass obsoletion

Replaces #5870 (although the procedure is slightly different)
Workflow: https://github.com/cmungall/mondo-refactoring
Corresponds to these artefacts:
https://github.com/cmungall/mondo-refactoring/tree/77b5f39bc149290f3d43eef0c07c614d61d602bb

intersection of ordo-only classes and ancestors of omim-equivs
some terms cannot be obsoleted until logical-defs referencing them are removed

I looked at the first one:
FAIL Rule ../sparql/qc/mondo/qc-omim-subsumption.sparql: 38 violation(s)
entity,property,value
http://purl.obolibrary.org/obo/MONDO_0007078,http://www.w3.org/2000/01/rdf-schema#subClassOf,http://purl.obolibrary.org/obo/MONDO_0019182
It looks like this is just revealing already present issues? The QC check should check for transitive subclasses. The rewiring just revealed an existing issue. http://purl.obolibrary.org/obo/MONDO_0019182 is a grouping for a polygenic trait https://omim.org/entry/601665

Replaced by #6447


Build failed - failed to run custom build command for openssl-sys v0.6.4

After successfully installing all the dependency that run.sh asked for and running run.sh again, I get this response:
(...)
   Compiling hpack v0.2.0
Build failed, waiting for other jobs to finish...
failed to run custom build command for `openssl-sys v0.6.4`
Process didn't exit successfully: `/home/kuba/temp/Eve/runtime/target/debug/build/openssl-sys-765ddf9de3c5179c/build-script-build` (exit code: 101)
--- stdout
cargo:rustc-link-lib=dylib=crypto
cargo:rustc-link-lib=dylib=ssl
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
CARGO_MANIFEST_DIR = Some("/home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/openssl-sys-0.6.4")
OUT_DIR = Some("/home/kuba/temp/Eve/runtime/target/debug/build/openssl-sys-765ddf9de3c5179c/out")
OPT_LEVEL = Some("0")
PROFILE = Some("debug")
debug 0
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
running: "cc" "-O0" "-c" "-ffunction-sections" "-fdata-sections" "-g" "-m64" "-fPIC" "-o" "/home/kuba/temp/Eve/runtime/target/debug/build/openssl-sys-765ddf9de3c5179c/out/src/openssl_shim.o" "/home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/openssl-sys-0.6.4/src/openssl_shim.c"

command did not execute successfully, got: exit code: 1

--- stderr
/home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/openssl-sys-0.6.4/src/openssl_shim.c:1:26: fatal error: openssl/hmac.h: No such file or directory
 #include <openssl/hmac.h>
                          ^
compilation terminated.
thread '<main>' panicked at 'explicit panic', /home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/gcc-0.3.11/src/lib.rs:514
stack backtrace:
   1:     0x7f71a300ea6e - sys::backtrace::write::h7f613ce09e7b1d1d5As
   2:     0x7f71a30173f6 - panicking::on_panic::he6322b413efd100fGFx
   3:     0x7f71a2fd79ce - rt::unwind::begin_unwind_inner::h16a4850d896eee23R6w
   4:     0x7f71a37aeb1c - rt::unwind::begin_unwind::h17421284490640175071
                        at ../src/libstd/rt/unwind/mod.rs:236
   5:     0x7f71a37a8140 - fail::hf0d4f1ca6d7a5404COa
                        at /home/kuba/temp/Eve/runtime/<std macros>:3
   6:     0x7f71a37b7ea3 - run::h7be3ffcf524eddceKHa
                        at /home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/gcc-0.3.11/src/lib.rs:485
   7:     0x7f71a37b1a2a - Config::compile_object::h661499c1644adb0aema
                        at /home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/gcc-0.3.11/src/lib.rs:276
   8:     0x7f71a37ab5ea - Config::compile::hd820e5738d396b0eRha
                        at /home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/gcc-0.3.11/src/lib.rs:233
   9:     0x7f71a377b079 - build_openssl_shim::h4ca03af19d6db933Sfa
                        at /home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/openssl-sys-0.6.4/build.rs:75
  10:     0x7f71a3774a33 - main::h53a555fd2c70138diaa
                        at /home/kuba/.multirust/toolchains/nightly-2015-08-10/cargo/registry/src/github.com-0a35038f75765ae4/openssl-sys-0.6.4/build.rs:65
  11:     0x7f71a3016c5d - __rust_try
  12:     0x7f71a30191ca - rt::lang_start::h996e83640adddea1BAx
  13:     0x7f71a3782f0b - main
  14:     0x7f71a2751a3f - __libc_start_main
  15:     0x7f71a37733b8 - _start
  16:                0x0 - <unknown>

We're investigating if we can remove the sub-dependency on OpenSSL, since we don't use it directly, nor do we use any features which rely on it's existence. In the interim, you can investigate using steve's suggestion to link against an existing OpenSSL installation. https://github.com/witheve/Eve/issues/190#issuecomment-131980782

I think this will fix it for the linux folks:
apt-get install openssl-dev

Unfortunately openssl is not an optional feature in rust-websocket, so removing the dependency is not as simple as I hoped.

@ibdknox I get
E: Unable to locate package openssl-dev

Should I add a specific ppa repository?

Ah sorry, it's libssl-dev:
sudo apt-get install libssl-dev

It helped, thank you!

Would it be possible to add this as a test in one of the build scripts. a check to make sure all pre-reqs are installed? I just ran into this as well.

Hey @thomas-shares, this isn't actually a prereq of Eve (or even of the rust crate that's pulling it in -- it's supposed to be able to build it's own local version if it can't find a system version). We're looking into statically linking our own version of SSL in to prevent this from cropping up in the future, since it's build chain for SSL seems to be very flaky.

Hi @joshuafcole ,
Thank you for your response. In my opinion if the build fails because something isn't installed, that makes it a pre-req :)
If it actually not needed or subsequently used it should be removed. Now I understand that this is easier said than done, so it might take more effort to remove it compared to adding a test to make sure it is there. I would be happy with either way, what ever is easiest for you guys.
And thank you for this exciting new product you guys are working on, it looks really interesting.

Ah, sorry I wasn't clear. I meant to say that a) we don't use it, but also (and more importantly) b) the rust crate that depends on it has a build script to compile and link against it's own internal version of OpenSSL in the event that there isn't a system one installed. So nobody should have to have a system version installed, the issue is just that the crate's build script is flaky.
Rather than requiring users to install and maintain their distro's version of OpenSSL, we're planning to statically link it to a binary we create in the next major version so we don't have to rely on the faulty build script anymore.
Cheers,
Josh

ok sounds good. thank you.


Re-use reqwest::Client and allow users to customize it

Creating new reqwest::Client on each request prevents reqwest from leveraging its connection pool, leading to unnecessary TCP handshakes.

Hi @TmLev , nice optimization by leveraging connection pool, thank you for the PR!
I'll release this sometime later this week/weekend.

This is released in v0.9.5


Allow services summary truncation to be configurable

Since we had similar configuration option for teams summary, I felt this would be a nice option to be configurable as well for services summary.

Just out of curiosity, could you help us understand why the builds failed. I was unable to figure it out.

It was an unrelated issue introduce 2 days ago in this commit - https://github.com/zerostaticthemes/hugo-serif-theme/commit/95aa132ee1441c65cf67dba2e345900748f77959.
For some reason the line break in the 404 page was crashing Netlify builds but not when running hugo locally. Very odd.


New "Lookup Rollup Summaries Tools" tab

This will be a new home page containing the following Lightning Components

News and Features A list of notable things added each release and any associated actions to be taken.
Optimizer Reviews the tools configuration (including possible misconfigurations), rollups, jobs, logs etc and makes recommendations that might require attention.
Jobs Review the apex jobs and schedules run by the tool shows time taken to execute. In the future will allow easier manageability of the scheduled jobs instead of per rollup.
Profiler Enable capture of SOQL queries made by the tool, then run the SOQL profiler over them to determine if indexes are being used or not. Results will be displayed and can then be shared with Salesforce Support to aid in enabling indexes and/or changing rollup criteria.

Sneak preview of the Optimizer...

Other planned checks are

Triggers deployed with no rollups
Rollups with Calculate Scheduled and also Full Calcualte Scheduled
Schedule Items with errors
Log entries
Calculate job records when no corresponding running job (calc job killed or fatal error)

Fixed in v2.12


Check the dictionary before cloning a habtm or has many through

deep_cloneable doesn't dup HABTM associated objects, only the associations themselves.
This patch introduces logic to check the dictionary for an existing clone.  This way if two classes joined by a HABTM (or has_many :through) are deep_cloned via a shared parent, they won't be unnecessarily duplicated.
I couldn't think of a situation in which you wouldn't want to check the dictionary, so I figured it would be best to make it the default behavior rather than passing in an option, but that's a possibility as well.
I have added a test to illustrate the use case.

Thanks, will look into it this week! Sorry for my late response, I was on holiday ;)

Great!
Yea, the call on like deep_cloneable.rb:148:
if(dict && object.find_in_dict_or_clone(dict, false))
Required it to be protected rather than private.


In version 3, authorization on the server does not work

Server version 3.6
Client 3.0
Returns an error no suitable server found at this address, but if you add a space at the end (https://rocket.chat/_) then the server finds.

I confirm, same in my case.

This issue seems to be a duplicate of some earlier filed ones. See  #1743


Lazy loading

I'm just wondering if it is possible to disable the prefetch on the install event, ideally for a particular file type.  I cache all of my .svg image files with the service worker, but on the install event the prefetch requests exceed the rate limit on my http server.  In my case it's not really necessary that the images are prefetched, and I would like to keep the rate limit intact.
Best,
Kyle

Nevermind, I see the optional property for the caches config.  Thanks anyway, and awesome plugin!


Continue the cleanup

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 16 (0.0%)  changed or added relevant lines in 2 files are covered.
14 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.03%) to 37.692%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/Kendrick/KendrickModelingComponent.class.st
0
7
0.0%

src/Kendrick/Visualization.class.st
0
9
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/Kendrick/Visualization.class.st
1
0%

src/Kendrick/KendrickModelingComponent.class.st
13
0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
5488

Relevant Lines:
14560

💛  - Coveralls


issue with exception

fix issue with folder/file checking at initializing the software, it through unnecessary exception.

Solved


flatten docs categories using collections

Changes
This PR does 3 things:

f98f67f: Uses Astro content collections to make it easier to query pages.

Frontmatter layout does not need to be set anymore, as it will be manually used in [...slug].astro.
Frontmatter group only needs to be set for "intro" and "utilities". All others are assumed to be "components".

2ad7f8d: Fixed LeftSidebarContent to use flattened categories (except "Utilities", as discussed)
02cb06d: Uses htm to reduce some of the repetition in LeftSidebarContent.

Testing
Tested website locally to ensure all categories are showing correctly, all urls are unchanged, and links are working.
Docs
N/A

@siddhantrawal That delay is unrelated and only happens locally (even before this PR), because it is running the code on demand. In the deployed prod version, the html has already been generated ahead of time so there is no delay.


add option for location of prefs and temp files #2779

--portable
puts what normally goes in ~/.beaker instead into the folder where the application is mounted - {path_to_beaker_application}/.beaker .

:+1:


better check for reset crash

Should kbctx be protected with a lock ?
On Sat, Nov 9, 2019 at 14:17 Mike Maxim<EMAIL_ADDRESS>wrote:

Merged #20886 https://github.com/keybase/client/pull/20886 into master.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/keybase/client/pull/20886?email_source=notifications&email_token=AAIXJVD5VUVGQQO7NHXAYT3QS4EGDA5CNFSM4JLJFITKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOUYEDLPY#event-2785555903,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIXJVCZZYODTMZQKV5ZSITQS4EGDANCNFSM4JLJFITA
.


openapi: support nullable attribute with temporary generator image

Q
A

Bug fix?
no

New feature?
yes

API breaks?
no

Deprecations?
no

Related tickets
https://github.com/OpenAPITools/openapi-generator/pull/1869

License
Apache 2.0

What's in this PR?
The OpenAPI specification gives support for nullable fields via the nullable attribute but this feature has not landed yet in the official generator CLI, thus I have updated the image built from the PR branch until the feature arrives.
Why?
It's not possible to use pointer fields for example *time.Time is not achievable, however null and time.Time = 0 is very different.
Additional Context
oneOf was broken (made no effect on the generated code, it still generated map[string]interface{}) previously as well, but now it becomes broken (the hard way) since it references a type called OneOfAorBorC but that actual type is not generated (results in compile time error). For the time being, I have commented out the broken oneOf parts from the API specification.
Checklist

[ ] Implementation tested (with at least one cloud provider)
[ ] Error handling code meets the guideline
[ ] Logging code meets the guideline (TODO)
[x] OpenAPI and Postman files updated, client regenerated (make generate-client) (if needed)
[ ] User guide and development docs updated (if needed)
[ ] Related Helm chart(s) updated (if needed)

FYI:  https://github.com/OpenAPITools/openapi-generator/pull/1869 has been merged into master. 😉

Thanks for the heads up @ackintosh !


Upgrade to operator-sdk 1.22.0

Following steps from https://sdk.operatorframework.io/docs/upgrading-sdk-version/v1.22.0/

Upgrade Kubernetes dependencies from 1.23 to 1.24
Upgrade controller-gen from v0.8.0 to v0.9.0

Also adding back the dependency to controller-tools in go.mod
NOTE: the bump of golang from 1.17 to 1.18 was done already, because of other dependencies - no further change here.

/ok-to-test


[Enhancement] keep fe plan test stable by disabling checking tablets

What type of PR is this：

[ ] bug
[ ] feature
[x] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

[FE PR Coverage check]
:heart_eyes: pass : 0 / 0 (0%)

@Mergifyio backport branch-2.3


Completed Tasks

Enhancement description
I would like to get my completed tasks. It is available via the API. I don't know why I can't access them with this API.

Is this being worked on? For example get_completed_tasks()? Or get_tasks(is_completed = "True")?


wasm: fix invoice generation

The InvoiceBuilder::current_timestamp uses SystemTime::now() internally, which is unimplemented on wasm and thus panics.
We can use getTime on new Date() on wasm.
This is not done in std library because wasm32-unknown-unknown can't garauntee presence of a JS runtime.

Codecov Report

Merging #430 (9f76939) into master (3606a20) will increase coverage by 0.03%.
The diff coverage is 100.00%.

:exclamation: Current head 9f76939 differs from pull request most recent head 7a92c03. Consider uploading reports for the commit 7a92c03 to get more accurate results

@@            Coverage Diff             @@
##           master     #430      +/-   ##
==========================================
+ Coverage   74.52%   74.55%   +0.03%
==========================================
  Files          83       83
  Lines       10515    10518       +3
==========================================
+ Hits         7836     7842       +6
+ Misses       2679     2676       -3

Impacted Files
Coverage Δ

client/client-lib/src/lib.rs
88.92% <100.00%> (+0.05%)
:arrow_up:

minimint/src/net/connect.rs
95.20% <0.00%> (+0.25%)
:arrow_up:

minimint/src/net/peers.rs
93.43% <0.00%> (+0.52%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Top level async/await doesn't work in JavaScript runtime

When running JS code in the Sandbox, the top level promise is never awaited:
import { Sandbox } from '@e2b/code-interpreter'

const sandbox = await Sandbox.create({apiKey: ''})

const code = `
 const printResult = async () => {
      return 'foo';
    };

  printResult()
    .then((result) => {
      console.log("glama.result");
    })
    .catch((error) => {
      console.log("glama.error");
    });
`

console.log('running code')
const ex = await sandbox.runCode(code, {
  language: 'js',
  onError: (error) => {
    console.log('error', error)
  },
  onStdout: (stdout) => {
    console.log('stdout', stdout)
  },
  onStderr: (stderr) => {
    console.log('stderr', stderr)
  },
})
console.log('code executed', ex)

The code above will produce a following output
running code
code executed Execution {
  results: [
    Result {
      isMainResult: true,
      text: 'Promise { <pending> }',
      html: undefined,
      markdown: undefined,
      svg: undefined,
      png: undefined,
      jpeg: undefined,
      pdf: undefined,
      latex: undefined,
      json: undefined,
      javascript: undefined,
      raw: [Object],
      data: undefined,
      chart: undefined,
      extra: {}
    }
  ],
  logs: { stdout: [], stderr: [] },
  error: undefined,
  executionCount: 2
}

The text field says 'Promise { <pending> }' which means the promise was never awaited.

There is no top-level await in this code by the way.
Anyway, is there a workaround – I just need to call a function that results Promise and get that response somehow.

There is no top-level await in this code by the way.

I should have clarified. There's a top level promise that isn't ever awaited.

@punkpeye we're adding a new JS kernel in this branch https://github.com/e2b-dev/code-interpreter/tree/change-js-kernel

Says not-found.
Also, that Kernel worked completely fine. Looks like no one just read the documentation.

Says not-found.
Also, that Kernel worked completely fine. Looks like no one just read the documentation.

It got merged - https://github.com/e2b-dev/code-interpreter/commit/5894232f2360c1462bc099cdaeaec6902fca3b44
Still in preview though


fix: fetch entries of all CTs when required by an intent

When an intent returns true for requiresAllEntries we need to skip the content type scoping of entries we fetch, and fetch all existing entries out there.
This was accidentally broken during https://github.com/contentful/contentful-migration/pull/160
Now it is covered by unit tests too so we hopefully can avoid this regression in the future.

:tada: This PR is included in version 0.16.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


OSError: Could not find a suitable TLS CA certificate bundle

I have pve-exporter deployed in a k3s cluster, but it can't scrape from proxmox. I'm using the docker image that shown on the readme (prompve/prometheus-pve-exporter:latest). It appears to be something to do with verifying https certs, but I have disabled ssl verification so I'm confused about why it would even be getting to a step of checking certificates. This is a complete log:
[2023-05-11 04:48:35 +0000] [7] [INFO] Starting gunicorn 20.1.0
[2023-05-11 04:48:35 +0000] [7] [INFO] Listening at: http://<IP_ADDRESS>:9221 (7)
[2023-05-11 04:48:35 +0000] [7] [INFO] Using worker: threads
[2023-05-11 04:48:35 +0000] [8] [INFO] Booting worker with pid: 8
Exception thrown while rendering view
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/pve_exporter/http.py", line 96, in view
    return view_registry[endpoint](**params)
  File "/usr/lib/python3.10/site-packages/pve_exporter/http.py", line 38, in on_pve
    output = collect_pve(self._config[module], target, self._collectors)
  File "/usr/lib/python3.10/site-packages/pve_exporter/collector.py", line 332, in collect_pve
    return generate_latest(registry)
  File "/usr/lib/python3.10/site-packages/prometheus_client/exposition.py", line 198, in generate_latest
    for metric in registry.collect():
  File "/usr/lib/python3.10/site-packages/prometheus_client/registry.py", line 97, in collect
    yield from collector.collect()
  File "/usr/lib/python3.10/site-packages/pve_exporter/collector.py", line 45, in collect
    for entry in self._pve.cluster.status.get():
  File "/usr/lib/python3.10/site-packages/proxmoxer/core.py", line 169, in get
    return self(args)._request("GET", params=params)
  File "/usr/lib/python3.10/site-packages/proxmoxer/core.py", line 144, in _request
    resp = self._store["session"].request(method, url, data=data, params=params)
  File "/usr/lib/python3.10/site-packages/proxmoxer/backends/https.py", line 229, in request
    return super().request(
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 460, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 263, in cert_verify
    raise OSError(
OSError: Could not find a suitable TLS CA certificate bundle, invalid path: false

I've tried to disable ssl verification with the config variable verify_ssl and also the environmental variable PVE_VERIFY_SSL, both with the same result.
I really appreciate any help you can provide.

Thanks for taking the time to report this issue.
This looks like it is coming from the python requests package. Regrettably I cannot reproduce this problem. What happens when you run the image in docker or podman?

Could you try setting PVE_VERIFY_SSL to 0 (instead of false)? It looks like the requests module does something funny when it thinks that the verify param is a string (not a boolean):
https://github.com/psf/requests/blob/d3d504436ef0c2ac7ec8af13738b04dcc8c694be/requests/adapters.py#L253-L258

Could you try setting PVE_VERIFY_SSL to 0 (instead of false)? It looks like the requests module does something funny when it thinks that the verify param is a string (not a boolean):

afaik, environment variables are always strings. I tried to set it to 0 instead of false and I got a different error. This looks like its attempting to verify the certs:
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/pve_exporter/http.py", line 96, in view
    return view_registry[endpoint](**params)
  File "/usr/lib/python3.10/site-packages/pve_exporter/http.py", line 38, in on_pve
    output = collect_pve(self._config[module], target, self._collectors)
  File "/usr/lib/python3.10/site-packages/pve_exporter/collector.py", line 316, in collect_pve
    pve = ProxmoxAPI(host, **config)
  File "/usr/lib/python3.10/site-packages/proxmoxer/core.py", line 210, in __init__
    self._backend = importlib.import_module(f".backends.{backend}", "proxmoxer").Backend(
  File "/usr/lib/python3.10/site-packages/proxmoxer/backends/https.py", line 302, in __init__
    self.auth = ProxmoxHTTPAuth(
  File "/usr/lib/python3.10/site-packages/proxmoxer/backends/https.py", line 59, in __init__
    self._get_new_tokens(password=password, otp=otp)
  File "/usr/lib/python3.10/site-packages/proxmoxer/backends/https.py", line 70, in _get_new_tokens
    response_data = requests.post(
  File "/usr/lib/python3.10/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='<IP_ADDRESS>', port=8006): Max retries exceeded with url: /api2/json/access/ticket (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY
_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

This looks like it is coming from the python requests package. Regrettably I cannot reproduce this problem. What happens when you run the image in docker or podman?

It worked in docker. I did run into an authorization error which I fixed in the configuration. I tried to use that configuration with the one in kubernetes, and only specified that it shouldn't verify ssl in the config file, and it worked! I think since I had it as an environmental variable AND a config value and something was not happy.
Thanks for the help!

:+1:

Happened to come to this issue from Google while facing a similar issue. I'm running in Kubernetes and using env vars to configure the connector. It does seem like there is some an issue with env vars configuring PVE_VERIFY_SSL, same as above basically, it's (probably) a string, and needs to be converted to a bool for the env vars to work.


[CdkScrollable] Add more documentation and demo examples

i'm trying to put together the test of how to put a "scrollspy" together.
but unfortunately i did not find any help in the documentation.
can you expect a user manual?
thanks!

+1

+1

+1

监听的时候难道这个事件飞到了window 层？试过很多办法。数据绑定都是失效的。

监听滚动事件的时候，数据绑定是失效的。
此时比较折中的办法应该是 对 元素直接操作，代码触发angular 的数据视图更改可能会有潜在的风险。

Example with Sidenav
https://stackoverflow.com/a/50812763/1071200

I'm struggling to implement the CdkScrollable feature. Please provide more examples.

wanted demo for scroll spy, anyone?


Support for column spanning

Title says it all. Similar to https://github.com/tj/terminal-table .

Hi Myles 👋
Thanks for using tty-table. I'm in favour of this feature. Do you have time to contribute PR?

Hi Myles 👋
Thanks for using tty-table. I'm in favour of this feature. Do you have time to contribute PR?

Hi @piotrmurach. Not soon, but it's possible in the mid/mid-long future. I'll see if I can fit it into my overly-busy life.

Hi @piotrmurach. Not soon, but it's possible in the mid/mid-long future. I'll see if I can fit it into my overly-busy life.

Hi @piotrmurach. Not soon, but it's possible in the mid/mid-long future. I'll see if I can fit it into my overly-busy life.


r.url("https://www.baidu.com")

import rpa as r
r.init()
r.url("https://www.baidu.com")
r.wait()
r.close()
可以正常打开浏览器到首页但是打不开指定的页面  希望大佬指点

Hi @luSir997 can you tell me more?
I am able to open and go to Baidu webpage using your code above.

Closing this issue for now but will look out for any reply


Latest translations

Latest translations for React-Spectrum, for all new strings as well as general linguistic bug fixing
PR is complete and ready to merge if looks okay,

GET_BUILD

@devongovett I've updated for comments above, no more changes planned, please let me know if anything else, and we can review locale support separately, (related to sr-SP above)


Allow scraping Prometheus operator CRDs across all namespaces

Hey!
Loving the chart so far. I have just noticed that there is an inconsistency between settings related to capturing and forwarding logs and the settings related to capturing and forwarding metrics.
The default values in helm chart for log configuration for grafana agent assume the following:

we scrape logs from all namespaces (via discovery: "all")
There is namespaces: [] option that does -> Only capture logs from pods in these namespaces ([] means all namespaces)

Now when configuring metrics scraping via prometheus operator CRD, by default we have:

scraping for podMonitors, probes, serviceMonitors
In each we can find namespaces: [] option that is described as -> Which namespaces to look for Probe objects.

Because of the way all of the prometheus operator CRDs are currently templated (for example: https://github.com/grafana/k8s-monitoring-helm/blob/main/charts/k8s-monitoring/templates/agent_config/_prometheus_operator_objects.txt#L4). There is no option to scrape from all of the namespaces (one has to pass each one that should be scrapped explicitly). I would like to scrape all of them across all of the namespaces (which can be supported by removing the namespaces block from the river config).
This has been probably made because many of the ServiceMonitors and other CRDs (common for Kubernetes monitoring mixin) are covered in this chart via river grafana agent configs. I was wondering wether such option could be implemented in this chart? That might be a breaking change tho, since the meaning of default empty list would be flipped and the agent would look across many namespaces. Also alternative ways of allowing such setting could be evaluated.
I am opening this issue in this helm chart since I really love some of the default river configs you guys provide and the support for scrapping prometheus operator CRDs from all of the namespaces would be really great + would match the default behaviour of logs scrapping.
Please let me know wether you would consider such feature and if not I would probably disable prometheus CRD scrapping and throw my own equivalent river config to extraConfig. The only issue with that is that currently extraConfig is not templated and therefore one can't depend on other helm values which would be really useful, for example to set default_scrape_interval like it's being set in other river configs. This would be a great feature too!

IIRC, even if the prometheus.operator.servicemonitors component has namespaces = [] in it, it'll look in all namespaces and not in no namespaces. My read of the code in the Agent matches that, since the loop will iterate over an empty list.
That being said, it's not altogether clear, so let me make a change to only include that if the list is non-empty.

https://github.com/grafana/k8s-monitoring-helm/pull/393

I have just done a quick reinstallation with the default prometheus operator CRDs scraping via the chart river config and it indeed works across all of the namespaces. Sorry for the trouble, I must have misconfigured something on my previous setup.
Nonetheless, thanks for the clarifications and the config cleanup! If you have chance please take a look at https://github.com/grafana/k8s-monitoring-helm/pull/394 which refers to my initial comment with adding the template functionality.


defaults in diff() dont work

This is from the documentation:
include <BOSL2/std.scad>
diff()
cuboid(50) {
attach(TOP) sphere(d=40, $tags="remove");
attach(CTR) cylinder(h=40, d=10, $tags="keep");
}
This crashes OpenSCAD because there is an undefine variable
include <BOSL2/std.scad>
diff("remove")
cuboid(50) {
attach(TOP) sphere(d=40, $tags="remove");
attach(CTR) cylinder(h=40, d=10, $tags="keep");
}
Does not crash but keep does not work
include <BOSL2/std.scad>
diff("remove","keep")
cuboid(50) {
attach(TOP) sphere(d=40, $tags="remove");
attach(CTR) cylinder(h=40, d=10, $tags="keep");
}
Works properly
OpenSCAD version 2021.01
BOSL2. ?version I downloaded it off github about April 20
Absolutely Brilliant library BTW

Defaults for diff() were added on April 24th.  Re-download the latest and it should work.


Improve to_array and to_tensor

Codecov Report
Patch coverage: 100.00% and no project coverage change.

Comparison is base (29dfc9d) 100.00% compared to head (72a941f) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #391   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           19        19
  Lines         1769      1775    +6
  Branches       183       185    +2
=========================================
+ Hits          1769      1775    +6

Files Changed
Coverage Δ

src/redcat/utils/array.py
100.00% <100.00%> (ø)

src/redcat/utils/tensor.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Utilizing paths feature without baseUrl

Hi!
I'm trying to figure out how to use the compilerOptions.paths behavior while retaining the default behavior of compilerOptions.baseUrl.
The following examples assume a folder structure like so:
my-app/
  tsconfig.json
  package.json
  src/
    index.ts
    something.ts

For example, if we want to support absolute imports via @/, we could configure paths like so:
{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "@/*": ["src/*"]
    }
  }
}

Unfortunately, this has the side effect of permitting import 'src/something'; since baseUrl is set to ..
It appears as if you can replicate the desired behavior by using the following configuration:
{
  "compilerOptions": {
    "baseUrl": "node_modules",
    "paths": {
      "@/*": ["../src/*"]
    }
  }
}

However, I'm not sure if this is functionally equivalent to the following example for all intents and purposes (assuming this worked):
{
  "compilerOptions": {
    "baseUrl": null, // undefined, whatever
    "paths": {
      "@/*": ["./src/*"]
    }
  }
}

Can you confirm if baseUrl: 'node_modules' is identical to baseUrl: undefined? Or are these behaviors different?
Thanks!

Also wondering about this.


Prisma Client Go will not be Officially Maintained any more

Hey everyone,
After a lot of consideration, we've decided to stop the development on the Go Client. The Go client has unfortunately not seen the growth we were hoping for. To understand this decision a bit better, I'm sharing a chart that shows the number of installs over the last 12 months:

As you can see, there's been stable adoption, but not much growth.
This doesn't mean that the Go client isn't useful or that the Go Client is a bad idea. We've just learned that the Go Client won't grow by itself. We would need to invest more resources to reach the kind of adoption we would need to sustain the development. Given our priorities for the next year, the Go Client just didn't make the cut.
I'd like to take a moment to thank @steebchen for his work on the Go client for the last 2 years. He's been working tirelessly on answering questions, fixing bugs and implementing the features you've been asking for.
I've added an FAQ below, but I'll leave this thread open to answer any additional questions you might have. We're sorry for the inconvenience this may cause you.
FAQ
When does the development stop?
We are stopping the development of the Go Client immediately.
Will critical issues and bugs be fixed?
Unfortunately, we won’t have any resources to fix any outstanding or upcoming issues with the Go Client. Community members are more than welcome to fork the repo and continue the development of the Go Client on their own (as a reference, you can look at the Python Client which has been developed from scratch by a community member as well). If anyone is interested in that, we are happy to help with the first steps in adopting and distributing.
What is going to happen to the repo?
This repository will remain in place as a point of reference.
Will you change your mind in the future and pick up the development again?
Possibly. As we grow larger and reach more enterprises, the Go ecosystem is likely to become more relevant again. The type of Go Client we offer then may look different than the current one though.

Cool, right after I learned about prisma go client. 🙄

This is super unfortunate, you guys made an amazing ORM :(. Thanks @steebchen for all your work maintaining the client, especially the speedy issue responses and quick fixes

That is unfortunate. But isn't it also a bit premature to make any conclusions on adoption with something that has only been in "Early Access"? The note on those kinds of features are "We don't recommend using Early Access features or products in production."
So for instance I have personally followed the development and wanted to use this when it was ready, but I don't have time to play with it until it is more ready.
I of course respect your decision on this, but I find the argument a bit weak. I also would not expect to see a lot of growth for something that you can't use for anything real :)

Oh No... I've been struggling for a while whether I should migrate to Prima (from gorm).
Prima really accords with my ideal ORM.  None of the other ORM in Go (as I know)  can construct Where clauses with auto completion and provide consistent development experience.
The reason that Prisma haven't seen the growth is probably nobody knows it. If I am a beginner, I will Google it to compare all the ORM. And no results in the first page (for my locale, key word: go mysql orm and go postgres orm).

I found Prima because I use Nest.js too, and I found Prima in nest.js document. I haven't started using it yet (because I use MongoDB for node) but it impressed me a lot. When I use Go for MySQL and Postgres, I am disappointed to find that the popular ORM in Go fail to provide a good experience(compared with TypeScript, the latter provide a really good type system). So I just wonder will some library provide SQL generator just like Prima, and I found that Prima offers it itself! However, when I hesitated whether to migrate to Prima, I found this announcement.
Anyway, what you guys are doing is awesome, and hopefully more people will get a chance to use it one day.

This is really unfortunate to hear, thanks @steebchen for maintaining it and answering all my questions over the years!
I literally just migrated my company's database ops to Prisma, and while Prisma itself is still useful as a schema and migration tool we'll have to consider an alternative.
Which sucks a lot because I've been writing Go for many years and none of the database connectors come anywhere close to Prisma - they're all string based and not type safe.
I may consider helping maintain this as it's so useful and I use it in 3 production applications (2 of which are businesses)
I am also surprised at the move, but I think this tool really needs more fanfare in the Go community. I attend the London Gophers meetup and this would be a fantastic platform to tell Go engineers about Prisma. I really hope you focus on Go again in future as it's a huge language in the enterprise space and in my experience, often the go-to tool for migrating legacy systems into new world tech - prisma db pull is vital for these legacy migrations.
Anyway, thanks for your work! I will poke around the codebase and learn it a bit more.

This is really unfortunate and is a bit of a painful reminder of the abandoned Nexus Framework project and the abandoned nexus-plugin-prisma package (that one with a new project to rebuild it, but with zero support of the old one in the meantime and putting people who were using it in production in a horrible position for an incredibly long time now). Both of these totally screwed us for lengthy stretches.
I'm not sure if this is being unfair to Prisma or not, but it's starting to feel like there is a bit of a grave yard that we keep arriving at time and time again...
In any case, echoing other comments, thanks so much @steebchen for all of your work on this – I can't imagine how you must be feeling with this news.

Thanks @steebchen for your great work.
It's a bad news for those who are migrating their tech ecosystem from nodejs-prisma to golang-prisma or planning golang with prisma like me.  :(

From what I've heard from Go developers I know, the reason this project wasn't picked up was because it was branded as "experimental" so you can't really expect real usage at real live company software to use an experimental package.
But from my ~2 years of using it in 4 projects (2 of which were for business - yes, I'm a risk taker!) it didn't seem that unstable and I feel like you could have made the pitch a little less scary to increase usage.
Another thing is just the age old blog posts and videos/screencasts to get people excited - the TypeScript side of Prisma has loads of great marketing efforts. If the same were to be done for Go, believe me it would be huge because Go's database landscape is really really lacking compared to more mature languages like Java, C#, Ruby, etc.

Just want to come at this from another angle.
I tried picking this up myself and kept running into issues.  So went back with plain old database/sql.
I'm practically on the verge of dropping Prisma entirely.  Don't get me wrong, I love the typescript version.  But with the last 5 or 6 releases being focused on Mongodb, which frankly is not taken seriously because Postgres with a jsonb column is 100x better.  It's signalling to many, that the desire to be cross compatible rather than shoring up important issues that the community (loudly) has been asking for, is more preferable.
With a barrage of issues that are open, not only affecting me and (probably) other individuals on the same boat after a full year.  My excitement is starting to wane.  I'm well past the point where basic features will suffice and wanting more complex functionality just isn't there, well is frankly disappointing.
Upon hearing these news, well its saddening.  You don't develop a library in the hopes that it will be popular.  With Go, you develop it because you want to be in the Enterprise space.  And that's what you want to signal.

Just want to come at this from another angle.
I tried picking this up myself and kept running into issues.  So went back with plain old database/sql.
I'm practically on the verge of dropping Prisma entirely.  Don't get me wrong, I love the typescript version.  But with the last 5 or 6 releases being focused on Mongodb, which frankly is not taken seriously because Postgres with a jsonb column is 100x better.  It's signalling to many, that the desire to be cross compatible rather than shoring up important issues that the community (loudly) has been asking for, is more preferable.
With a barrage of issues that are open, not only affecting me and (probably) other individuals on the same boat after a full year.  My excitement is starting to wane.  I'm well past the point where basic features will suffice and wanting more complex functionality just isn't there, well is frankly disappointing.
Upon hearing these news, well its saddening.  You don't develop a library in the hopes that it will be popular.  With Go, you develop it because you want to be in the Enterprise space.  And that's what you want to signal.

This is horrible, and I just feel like someone has just pulled the rug under my feet... I have 3 production apps running on this client.
Now I have to spend days migrating 10'000s of LOCs to ent.

If you label a project as Early Access clearly people would hesitate adoption(a few of us took risks to run it on prod so that we can give feedback https://github.com/prisma/prisma-client-go/issues/539 and make the package better).
This is a punch on the face for some of us.
I'd like to thank @steebchen for his patience and swiftness in responding to my queries and issues and the amount of effort he put in to this project. ❤

I find this to be a really strange and anti-strategic decision for the Prisma team to make.
I thought Prisma's brilliant innovation was that it allowed you to have a single schema (or set of schemas), from which you could:
a. generate clients for every language you use
b. have a multitude of services sharing a single database or a set of databases
c. manage everything through a single interface and tool set (e.g. prisma migrate)
Such an innovation would solve problems I've had at every company I've ever worked for. Want to have two services written in different languages that use the same database? You can do that. Want to start writing things in language A and then port them to language B? You can do that too. Want to move a use case from service A to service B without having to rewrite everything? Basically zero effort.
The fact that the Prisma team is moving away from this work tells me they don't understand this enormous problem and the value that they were already providing in solving it. Instead of solving this problem (which is widespread, and for which I don't know of another existing equivalent solution) and leaning into it, the Prisma team seems to just be focusing on being a better node ORM than sequelize or knex or waterline or any of the already pretty good ORMs for node.
Definitely a head-scratcher.

Hey @johnpena, thanks for your message, I think your read is correct.

Prisma team seems to just be focusing on being a better node ORM than sequelize or knex or waterline

This is definitely our top goal for the next year or so. We want to be the dominate ORM in the Node.js ecosystem. We're not there yet. Reaching this milestone is required for everything else we do.

I thought Prisma's brilliant innovation was that it allowed you to have a single schema (or set of schemas)

I totally agree with you. As a Go and TS developer myself, being able to switch between the languages and apply the same concepts with the same schema is hugely useful.
This was seriously a tough call. But it boiled down to:

We really need to focus on making the TS Client the best it can possibly be
The Go Client isn't a breakout success and requires effort and resources to make it so (someone mentioned videos, content marketing, etc.)

I'd also say I think we started this effort too early in our journey. After we become the defacto standard in the TS ecosystem, I do see us branching out to other languages again, more deliberately, with more resources. How that shapes up, remains to be seen.

Do you plan to provide resources for community clients in future? I'd love to look into contributing and buiding generators but there seems to be a lack of documentation for the query engine RPCs.

I would like to recommend SQLBoiler as an alternative.
It generates type-safe models and query builders from your DB schema, and you can confidently regenerate the models whenever your DB schema changes.
This means you can continue to use Prisma's schema definition to manage your schema and migrations, and then generate your models from your updated DB.
Sadly, this does not really help those who are already using this client in production, migrating to any other tool would likely be a bit difficult, but for newer codebases, I believe SQLBoiler is a pretty great ORM with a similar philosophy that can fit well with other Prisma tools.
Disclaimer: I am one of the maintainers of SQLBoiler (although that is a recent development 😅)

I would like to recommend SQLBoiler as an alternative.

From SQLBoiler's documentation:

Table names and column names should use snake_case format.

We require snake_case table names and column names. This is a recommended default in Postgres

This means we would need to change all our names to snake_case to use SQLBoiler, right?

This means we would need to change all our names to snake_case to use SQLBoiler, right?

That is pretty old, and this is no longer true. I took some time to check this and it works just fine.
To be clear, postgres makes everything lowercase, so when getting the table names and column names from the DB to generate the schema, with postgres, the driver gets everything in lowercase which may cause the model attributes to look weird.
For example, Jets.PilotID will become models.Jet{Pilotid string}

This is very sad news, the Go Client was instrumental in the creation of my Python Client, it probably wouldn't even exist without the Go Client. I'd like to say a massive thank you to @steebchen 💚. I hope Prisma picks this back up in the future.

@Southclaws Do you plan to provide resources for community clients in future? I'd love to look into contributing and buiding generators but there seems to be a lack of documentation for the query engine RPCs

If anyone is interested in building generators I'd be more than happy to help and answer any questions.
For RPC reference these modules could be useful for anyone wanting to implement it themselves:
Generation RPC loop:
https://github.com/RobertCraigie/prisma-client-py/blob/main/src/prisma/generator/generator.py#L80-L133
RPC interface:
https://github.com/RobertCraigie/prisma-client-py/blob/main/src/prisma/generator/jsonrpc.py

I just thought about how convenient it would be to use Prisma for golang and I got excited when I found out there was a client library. This is sad news ... tbh.

Looks like SQLBoiler is the tool to go for. It plays nicely with Prisma's existing schema management.
I was evaluating Ent and sqlc but both are tightly coupled to the schema and migration process. I'd prefer to keep using Prisma's schema and migration tools as they are excellent.
Because SQLBoiler reads from the DB in order to generate code, this makes it possible to work alongside Prisma. I think more tools should be built around this database-first paradigm of generating stuff (like declarative source-of-truth schemas or code) from the database itself!
So, my workflow might look like:

Edit schema.prisma
Run prisma migrate dev --skip-generate skip generating code
Run sqlboiler generator from the newly migrated development DB
Avoid writing dirty non-typesafe awkward SQL string literals for another week 🎉 💯 😁

For those who loved using Prisma Go, I suggest taking a look at the Entgo ORM which is by far the best and the fastest in my opinion at least.

Though I was looking forward to Go support becoming GA, I completely understand and appreciate your decision to keep focused on providing the best experience for TS first.
It is far better to become the de-facto for a narrow market before expanding. I appreciate the work the team at Prisma has done so far on all of their offerings. I am very happy that you have the sense to make these difficult decisions because it means that one day (maybe) we can have this in Go!!
GO TEAM PRISMA!

Playing with Ent as an option, annoyingly it's a full solution (its own schema and migration tooling) so I started a simple converter script: https://github.com/Southclaws/prisment
Feel free to try it out, contribute code, etc.

Prisma is the bestest orm of golang and i've always believed. Just hope prisma for golang will be back in future someday. Thanks to every developer and contributor.

I can understand where @matthewmueller is coming from and I appreciate the clarity and the transparent communication from the Prisma Team. Things come and go, but support with Go will come in time. As @dfalgout said keeping focused on the best for TS first with prisma is the logical decision even though it's a tough decision.
I can empathize with @robin-macpherson on nexus-plugin-prisma support being dropped being similar to this but I think it was necessary and now the community can take it. For me using nexus-prisma is really smooth to use once you get past the JSDocs (still need work) but its low priority atm but its TS first so they'll hopefully keep support for that.
Go will come back it's just a matter of time and demand

Is it still useable? (I see you do some gradual updates)
I'm using both Node.js and Go and would love to see this project afloat...

Another person here who has been following the development of this for as long as I can remember. Was actually visiting this repository in the hope that it was finally stable.
Prisma was so close to becoming "the" data modeling language across not just my own personal projects, but at places I've worked. With this move, that is no longer feasible. Back to GraphQL and/or protobufs I suppose.
Thanks @steebchen for all of your hard work.

That's bad. I was starting to be encouraged to learn GoLang when I saw that The Great Prisma ORM had Support for Go.
Hopefully Things will change in the future. Great Job Guys !

I have a web app in TS and prisma and not playing to migrate to Golang, but when it counts to designing the CLI (terminal) I find better options in the Golang ecosystem. I would like to use my already defined prima schema to talk to the DB hence this project was extremely useful. I'm sad.

what can we do to bring this project back

I too would like for this to be supported again, prisma seems to be the de-facto ORM for TS/JS and it would be really nice to combine microservices written in go and TS using one schema definition and one client generator, would be incredible. Using sqlboiler for now

s to be the de-facto ORM for TS/JS and it would be really nice to combine microservices written in go and TS using one schema definition and one client generator, would be incredibl

The current pool of ORM clients as alternatives to Prisma is quite low in the Go ecosystem.

This is unfortunate, I am pretty sure if you release a GA version it soon will become the db package of choice in Go community. 'Early Access' keeps a lot of devs and organizations off from using it.

Used prisma in my node.js/typescript projects, moving to golang project would have liked this. Someone has mentioned ent but i found the introspection tool (entimport) is not as mature as prisma's.

I would also add that in my recent analysis of past few months, there is slowly a shift happening in the industry where more and more individuals and teams are realizing that node is not the way to go for backend applications and considering jumping ship to languages like Go and Rust. For example, you can see a recent post from Facebook Engineering about adding Rust as a supported server side language.
So not only would a Go client make sense but a Rust client as well.

I currently use Prisma for Node projects but now Go has become an important part of our stack, we wanted to use this Prisma client, and maybe we will if it gets enough community support. If I had to choose between keeping Prisma and switching to Go or Rust over Node, certainly I may have to drop Prisma from the stack, which would be unfortunate because I love Prisma!

Ent is almost there with Versioned Migrations but the DX is still not as good as Prisma was.

Ent is also a very nice project but (to my knowledge) it doesn't have a db pull equivalent, which is very helpful for migrating to prisma if you have an existing database.
It would also be great to have a solution that works in both Node and Go (and others, ideally). Take prisma with you to whatever language you use.
That being said, i've had a good experience with ent and it's worth checking out for anyone who's turned off by the unmaintained status of this lib.

@matthewmueller
Is there any update on this?

Ent is also a very nice project but (to my knowledge) it doesn't have a db pull equivalent, which is very helpful for migrating to prisma if you have an existing database.
It would also be great to have a solution that works in both Node and Go (and others, ideally). Take prisma with you to whatever language you use.
That being said, i've had a good experience with ent and it's worth checking out for anyone who's turned off by the unmaintained status of this lib.

They're not as good but there's primsment and entimport

They're not as good but there's primsment ...

Thanks @Southclaws. Hadn't seen these.
I actually started hacking on something similar the other day to do Prisma->Ent conversion. Even if prisma-go was maintained I think this might be preferable in certain cases, such as when you want to ship the whole thing to a user. You can do away with the prisma/engine and associated binary complexity.
Still hope prisma-go becomes a thing though.

For those still looking for an alternative, we moved our production code from prisma-go to sqlboiler. By and large this was a seamless transition (leaving aside of course the fact that we had to rewrite a lot of code.) sqlboiler models are generated from your database schema, so we've continued to use prisma for schema management and migrations.
One thing I feel the need to mention is that our database read and write speeds have dramatically increased, likely due to removing prisma's serialization/deserialization step.

Honestly, there are so many layers to this problem I'd rather just have a new relational database that doesn't need some 1970s text-based language to transpile to and was code-gen type safe from the start 😂

^ this 😂

Honestly, there are so many layers to this problem I'd rather just have a new relational database that doesn't need some 1970s text-based language to transpile to and was code-gen type safe from the start 😂

Something like SurrealDB?

Honestly, there are so many layers to this problem I'd rather just have a new relational database that doesn't need some 1970s text-based language to transpile to and was code-gen type safe from the start 😂

Something like SurrealDB?

Looks like this still uses an SQL-like query language, unless there's a more low level API hidden in their docs 👀

Late 2022, I started use Prima (maybe it's late). I think, no any tools better Prisma.
Anyway, thank you Prisma team.

Prisma having a first-class go toolset is a valuable proposition for adopters.
It provides a path for scaling projects that start in javascript/typescript but need to scale up parts/all of their API once they have stabilized. Leveraging a second, faster language, like go, gives them that escape hatch.
I hope to see this project picked back up.

This is one of the worse things I've read in a while. Imagine if any inventor based their work on a silly graph, we would still be using candles as opposed to light bulbs. We would still be using horses instead of vehicles.

It's frustrating I know but Prisma is a company with priorities. If they didn't make tough decisions like this, then the brilliant tooling Prisma have built may not exist! So while I think we're all disappointed that support has dropped so early into its lifecycle, we can be understanding that decisions must often be data-driven. I'm hoping Prisma make a return to Go once their business has scaled up a bit more and they have the capacity to commit resources to other markets. 🚀

I think what's missing in this data chart is the number of developers that don't adopt Prisma because of a lacking go client that is officially supported/maintained

I hope this project get started officially maintained it will be very easy to convert languages and sync between them

It's been a whole year and I'm very glad to see the features and adoption prisma(JS) got over this year. There are now thousands of tutorial videos and articles, a well-established community and QAs. I think now (after a complete year), the prisma team should reconsider their decision and continue work on go client. This will be an amazing gift for new year.

Here to suggest Bob as an alternative.
Bob has a first-class Prisma driver which will use your existing schema.prisma file to generate an ORM and Factory.
There is a known issue caused by the prisma CLI not passing @map and @db attributes on columns. See Issue 3998. However, if the community finds it useful, I plan to support the Prisma schema as closely as possible, writing a custom parser if necessary.
GitHub Link: https://github.com/stephenafamo/bob
Website: https://bob.stephenafamo.com/docs

Wow, I am actually shocked that Prisma would drop Go support entirely, right during the biggest growth phase of the entire Prisma project. This is a huge blow to the usability of Prisma in enterprise applications, especially ones that are shifting away from JavaScript due to performance and maintainability issues. Golang has huge adoption in enterprise environments, but is somewhat lacking in personal projects, which I feel is where the Prisma team is looking at specifically in this decision.
Please reconsider this decision, and at least provide a minimal client for Go, even if it may be somewhat lacking compared to the JS/TS client.

Well, I am pretty sad to hear that. Golang is for enterprises, not for home developers, looking for fancy interface tools.
We are a company developing services for enterprises, and we use java, go, nodejs, javascript, vuejs, flutter, k8s, etc.... etc.....
Golang is the by far the best utilized, best performing, stingy for resources, container friendly, easy to develop language we are using.

Please reconsider this.
On that note, I still see commits by @steebchen keeping things up to date.
Why not make it official? 😉

I guess this is another case of Go being widely adopted by application authors but not so much by library authors. Both the language and ecosystem have a bias towards YAGNI applications over building libraries.

Hello everyone, it's @steebchen, the main maintainer of the Go client.
I've been considering resuming work on the project, especially because of the recent activity. I truly believe that the Go client is a valuable tool that many people would appreciate, as the Go ecosystem currently lacks a robust database client and tooling at this time. Prisma seems to effectively fill this gap.
If anyone is interested in advancing the Go ecosystem by reviving the Go client, ensuring its maintenance, and enhancing it with new features, I would be sincerely grateful if you could consider supporting me through a sponsorship. Maintaining and expanding the Go client requires a significant amount of effort. The next big goal would be to release a production-ready version and market it as such.
I have some plans in mind for the Go client, but to dedicate more time to it than just a few hours a month, a sponsorship would be a wonderful option. By choosing a higher sponsorship tier, you would also receive the added benefits of prioritizing your issues as well as personal assistance from me, whether for you individually or for your company. I can provide help with setting it up, as well as consultation on any Go client, database, infrastructure, or CI/CD related matters.
If you're interested in supporting the development of the Go client, please consider sponsoring me via GitHub sponsors, or contact me to discuss. Your support would be greatly appreciated.

@steebchen lets remove the deprecation notice, love to see the Go client getting attention!

The Prisma Client Go repository was transferred to me now (big thank you, @janpio and @prisma!), so it lives at https://github.com/steebchen/prisma-client-go from now on, and is now officially maintained again by me.
---> Read the full announcement: Prisma Client Go: Resurrection <---
Read the quick start on the new docs page: Quickstart
If you already have a Prisma project, read the upgrade manual.
I'd also like to thank Alexander Belanger and Bryan Onel at https://www.oneleet.com for being the first bronze sponsor, as they are already using the Go client. If you like these news, let your employer know that the Go client is revived, and if they ask, send them my sponsor profile 😀
See --> https://github.com/steebchen/prisma-client-go/issues/860 for more!

Thank you @Oneleet, @f-tepel and @osilkin98 for sponsoring! ❤


Better format bracket

A plurality of nested brackets, ending a space reserved.
Input:
.foo {
  background: filter(radial-gradient(circle, #9ef, #0000af calc(100% - 20.38%)), hue-rotate(90deg))
}

Output:
.foo {
  background: filter( radial-gradient( circle, #9ef, #0000af calc(100% - 20.38%) ), hue-rotate(90deg) )
}

While adding an option, if you close this option, do not keep the space.

fmm.. I think, this is not need ;(

OK.

Thanks :)


Add models directory for old generators

This has been a long time coming. Closes https://github.com/feathersjs/generator-feathers/issues/94
I did some pretty extensive testing and I think this sets us up to have more configurable db options in the future.

This looks great, thank you! Dang, if I'd known I would've suggested to make those changes against the https://github.com/feathersjs/generator-feathers/tree/generator-refactoring branch (also see PR #159). It has some pretty extensive changes but is there any chance we can port this setup to that branch?

I almost finished making the changes against the new generators last night. There's a little more complexity to it, but I really like the updates.

This has been superseded by #162. Thanks again!