CustomProvider: ERR_UNSUPPORTED_ESM_URL_SCHEME

OS: Windows 10
node: 21.7.2
promptfoo: 0.51.0
When I try to use a custom provider (I think I have implemented it according to examples) then I get an error
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:239:11)
    at defaultLoad (node:internal/modules/esm/load:130:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:403:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:285:56)
    at new ModuleJob (node:internal/modules/esm/module_job:67:26)
    at #createModuleJob (node:internal/modules/esm/loader:297:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:254:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:235:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ModuleLoader.import (node:internal/modules/esm/loader:322:23) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

promptfoo.config

prompts: "{{question}}"
providers: [customProvider.js]
tests: vars.csv

const fetch = require('node-fetch');

class CustomApiProvider {
  constructor(options) {
    // The caller may override Provider ID (e.g. when using multiple instances of the same provider)
    this.providerId = options.id || 'custom provider';

    // The config object contains any options passed to the provider in the config file.
    this.config = options.config;
  }

  id() {
    return this.providerId;
  }

  async callApi(prompt) {
    // logic
}

module.exports = CustomApiProvider;

When in file node_modules\promptfoo\dist\src\esm.js I changed line const resolvedPath = path.resolve(modulePath) to const resolvedPath = 'file://' + path.resolve(modulePath).replaceAll('\\', '/');  then everything works correctly
On Linux this problem does not occur

Thanks for flagging. I believe this is addressed by
https://github.com/promptfoo/promptfoo/pull/652, which will be included in
a release published today. Hopefully that fixes it for you!
On Fri, Apr 12, 2024, 12:50 AM charles-m1 @.***> wrote:

OS: Windows 10
node: 21.7.2
promptfoo: 0.51.0
When I try to use a custom provider (I think I have implemented it
according to examples) then I get an error
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:239:11)
at defaultLoad (node:internal/modules/esm/load:130:3)
at ModuleLoader.load (node:internal/modules/esm/loader:403:13)
at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:285:56)
at new ModuleJob (node:internal/modules/esm/module_job:67:26)
at #createModuleJob (node:internal/modules/esm/loader:297:17)
at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:254:34)
at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:235:17)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ModuleLoader.import (node:internal/modules/esm/loader:322:23) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}
promptfoo.config
prompts: "{{question}}"
providers: [customProvider.js]
tests: vars.csv
const fetch = require('node-fetch');
class CustomApiProvider {
constructor(options) {
// The caller may override Provider ID (e.g. when using multiple instances of the same provider)
this.providerId = options.id || 'custom provider';
// The config object contains any options passed to the provider in the config file.
this.config = options.config;

}
id() {
return this.providerId;
}
async callApi(prompt) {
// logic
}
module.exports = CustomApiProvider;
When in file node_modules\promptfoo\dist\src\esm.js I changed line const
resolvedPath = path.resolve(modulePath) to const resolvedPath = 'file://' +
path.resolve(modulePath).replaceAll('', '/'); then everything works
corectly
On Linux this problem does not occur
—
Reply to this email directly, view it on GitHub
https://github.com/promptfoo/promptfoo/issues/662, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACLYJWEY2KOR2O3ISRWFBLY46G5JAVCNFSM6AAAAABGDVABAOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZTSMZSGYYDIOI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Fix released in 0.52.0


light-mode issues

[x] buttons on challenges not showing up in light mode
[ ] some images don't work well in light mode

maybe a global toggle to swap between light and dark mode

mistakenly closed, images need work.

superceded by #190


[Low] Bitfinex WebSocket API

I contacted Bitfinex regarding lifting my rate limits so I can trade on their platform.  They advised that WebSockets is basically limit free (as long as you have just one open socket connection).  I think it would be great if catalyst used the Bitfinex websocket API, this would fix all the ERR_RATE_LIMIT errors that occur (and that are quite common on your issues list).
I know this is major work and also perhaps the responsibility of CCXT but I thought I would add this as a future reminder.
Feel free to close if you feel its beyond the scopy of catalyst (i.e. CCXT responsibility)

Thanks @gatapia for opening this, we are always happy to hear from catalyst users which future features they would like to see. This is indeed on our roadmap.


[fix][test] Fix flaky test: PendingAckPersistentTest.testDeleteUselessLogDataWhenSubCursorMoved

Fixes #19437
Motivation
The beforeMethod doesn't have alwaysRun=true so it might happens that the setup is not called, especially in this case since the test class is in broker group and the test is in the quarantine one
Modifications

Set alwaysRun=true

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

/pulsarbot rerun-failure-checks


(chaos-exporter): Add Unit Test Cases for function getNamespaceEnv()

UNIT TEST
What happened:

Write the unit-test

What you expected to happen:

The test case should be written for function getNamespaceEnv() - https://github.com/litmuschaos/chaos-exporter/blob/master/cmd/exporter/main.go

Anything else we need to know?:

Issue related to Repository: https://github.com/litmuschaos/chaos-exporter
Reference:- https://github.com/litmuschaos/chaos-operator/blob/master/pkg/controller/chaosengine/chaosengine_controller_test.go

@Sanjay1611  @chandankumar4 May I take up this issue?


Inaccurate node capacity

I deployed InstaSlice (commit 38f4711) on a Kind cluster running on a node with 2 A100-40GB GPUs. The GPUs are preconfigured with one 3g.20gb slice on each. InstaSlice is not running in emulator mode, Kueue is not installed on the cluster. All the operator pods are healthy (gpu operator, instaslice operator).
I deploy 7 pods each requesting one 1g.5gb slice with the expectation that all these pods can run in parallel since there is enough GPU capacity for that.
I observe that the 7 slices are created as expected by InstaSlice:
GPU 0: NVIDIA A100-PCIE-40GB (UUID: GPU-31cfe05c-ed13-cd17-d7aa-c63db5108c24)
  MIG 3g.20gb     Device  0: (UUID: MIG-0f1cecc2-27a4-5452-85f2-ad9c3a15f1de)
  MIG 1g.5gb      Device  1: (UUID: MIG-c5720b34-e550-5278-90e6-d99a979aafd1)
  MIG 1g.5gb      Device  2: (UUID: MIG-5730a1b5-8b6c-5f12-bf06-7ddbca9f82c8)
  MIG 1g.5gb      Device  3: (UUID: MIG-d78b4033-6f19-5e4c-8f87-6ceec9e32825)
GPU 1: NVIDIA A100-PCIE-40GB (UUID: GPU-8d042338-e67f-9c48-92b4-5b55c7e5133c)
  MIG 3g.20gb     Device  0: (UUID: MIG-3dc2c68a-45e6-5acb-b043-caef296e6038)
  MIG 1g.5gb      Device  1: (UUID: MIG-48ee4d61-7ccb-5381-9feb-e2990edec372)
  MIG 1g.5gb      Device  2: (UUID: MIG-74680f0a-c3ce-5f76-9cda-be75b47b390b)
  MIG 1g.5gb      Device  3: (UUID: MIG-c147a260-9c75-5f09-a49a-8a9191af02d4)
  MIG 1g.5gb      Device  4: (UUID: MIG-3dea9fc8-d2d6-522a-bed5-f922795bbb36)

And the the conclusion of the test the 7 slices are destroyed as expected::
tardieu@netsres62:~/instaslice-operator$ nvidia-smi -L
GPU 0: NVIDIA A100-PCIE-40GB (UUID: GPU-31cfe05c-ed13-cd17-d7aa-c63db5108c24)
  MIG 3g.20gb     Device  0: (UUID: MIG-0f1cecc2-27a4-5452-85f2-ad9c3a15f1de)
GPU 1: NVIDIA A100-PCIE-40GB (UUID: GPU-8d042338-e67f-9c48-92b4-5b55c7e5133c)
  MIG 3g.20gb     Device  0: (UUID: MIG-3dc2c68a-45e6-5acb-b043-caef296e6038)

The reported node capacity however is not updated accordingly. After 30s, the node reports:
tardieu@netsres62:~/instaslice-operator$ kubectl get node kind-control-plane -o json | jq .status.capacity
{
  "cpu": "72",
  "ephemeral-storage": "958798960Ki",
  "hugepages-1Gi": "0",
  "hugepages-2Mi": "0",
  "memory": "1056712452Ki",
  "nvidia.com/accelerator-memory": "80Gi",
  "nvidia.com/mig-1g.5gb": "4",
  "nvidia.com/mig-3g.20gb": "2",
  "org.instaslice/30025a32-b37b-410f-ae65-41bdcd23008b": "1",
  "org.instaslice/36f449d9-540d-42c6-8ff4-24d10f4ea5cd": "1",
  "org.instaslice/410076ea-02c8-4aa7-8772-59f676f7aeec": "1",
  "org.instaslice/6558822f-aa09-4519-93f1-8c385063d432": "1",
  "org.instaslice/85a06967-78e5-4bb9-afd2-86cd958ecde8": "1",
  "org.instaslice/946306cc-dc82-4e43-8f9e-d00cfade4a84": "1",
  "org.instaslice/e8168de1-faa2-4d1c-a384-f768f7d778e5": "1",
  "pods": "110"
}

The reported 1g.5gb slice count is 4 instead of 7. Accordingly only 4 pods are successfully scheduled:
tardieu@netsres62:~/instaslice-operator$ kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
p1     1/1     Running   0          78s
p2     1/1     Running   0          78s
p3     1/1     Running   0          78s
p4     0/1     Pending   0          78s
p5     0/1     Pending   0          78s
p6     0/1     Pending   0          78s
p7     1/1     Running   0          78s

At the conclusion of the test, the reported node capacity remains stuck at 4 instead of 0:
tardieu@netsres62:~/instaslice-operator$ kubectl get node kind-control-plane -o json | jq .status.capacity
{
  "cpu": "72",
  "ephemeral-storage": "958798960Ki",
  "hugepages-1Gi": "0",
  "hugepages-2Mi": "0",
  "memory": "1056712452Ki",
  "nvidia.com/accelerator-memory": "80Gi",
  "nvidia.com/mig-1g.5gb": "4",
  "nvidia.com/mig-3g.20gb": "2",
  "pods": "110"
}

The reported slice counts vary from run to run.

This issue is similar to #96 but this time observed running in non-emulator mode aka with real gpus and the instructions to reproduce do not require kueue.

/cc @sairameshv @mamy-CS @asm582

Pending test case, I think this PR solves capacity update issue #108

Unit test added and test is green


Add validation to EEPROM highscore storage

A I loaded the game onto my Arduboy and love it alot! Works really great!
However, the title screen displayed me a highscore of somewhat around 70.000. Guess it just read some random EEPROM data from one of the previous games. In consequence, I implemented a validation mechanism :)
First, my additions add the string "Stellar_Impact" in front of the high score. Second, it adds a simple XOR of the high and low bytes, which should be sufficient to detect corrupted memory.
If either of the two checks fail, the highscore is reset to a default of -- well, 10, I took the freedom to change it from 100, but that's a simple #define we can change back if wanted.
Hope you like my change!
Felix

This is brilliant, thank you!


Lack username in the resetpassword confirmation email.

Either forgetting username or email will be asked to reset the password, but the confirmation email does not contain information about the username, which compulsory when logging in.
Thus if someone forget his/her username and reset the passwork via the link, he/she still could not get his/her username and cannot login.

Right, i think i didn't wire it up all the way for that. now in the UI it's only "forget email". but "forget username" is available via API. should change the wording on the UI

I just check....seems like it does send the username?
Dear Chan Tai Man,

Your username is standardUser.

You can reset you password by click the following link (active for 1 hour):

http://localhost:3000/resetPassword/standardUser/792a9611-446b-4c5c-b28f-c76f0dd81c42

Yours truly, NAME team.


Implement a "Rust" serializer

Serialize values to Rust code!
From IRC:

<onmyo> i wish serde had a 'rust' serializer

Justification: I think this would be a valuable example to showcase in the documentation.

The Rust serializer came out too long for an example so I went with a simplified JSON serializer instead.


Add support for Symfony 5

Ref #91
Two things were needed:

bypass the removal of the root_dir parameter and use the project_dir if available (a new parameter bugsnag.symfony_root was added);
add support for the new event classes.

I kept support for Symfony < 5 and Symfony >= 5.0 at the same time. I also added CI build for PHP 7.2 and PHP 7.3.
I also added an example application for Symfony 5.0.

Hello, can I have some feedback on this PR? We need Symfony 5 updates in several project but are blocked because this bundle is not yet compatible.
It looks like the tests are failing on PHP 5 because of a memory limit...

Hi @pyrech
We are actively working on testing this right now so we hope to be able to release in the near future.
Thanks!

Thanks :pray: :heart_eyes:

Hi @pyrech, we've pulled it onto a branch and created a PR at #93, which we'll test, review, and merged as soon as we can.
Thanks for the contribution!


[BUG] buy(), getProductInfo() errors everytime when used

EDIT: A NEW ALTERNATIVE ENDPOINT TO RESOLVE BUG AS EXPLAINED BELOW:
https://economy.roblox.com/v2/assets/ASSETIDHERE/details
Describe the bug
Global method buy(asset, price opt) errors everytime when used, citing:
HTTP Error 410: The page you requested was removed.
The stack track goes back to the getProductInfo() method, on line 43 which contains a request for (https://api.roblox.com/marketplace/productinfo), which when accessed via any browser returns the same error.
This leads me to believe that roblox has removed the endpoint, and now noblox.js needs to be updated to reflect this change. Personally hoping that roblox hasn't shut down all endpoints related to this and that there is still an alternative available.
To Reproduce
Log in via noblox.js and use the buy() method on any purchasable asset on roblox.
Expected behavior
A purchase request should be made successfully.
Additional context
Error: 410 The page you requested was removed.
at C:\Users\Crazyblox\node_modules\noblox.js\lib\asset\getProductInfo.js:43:20
at tryCatcher (C:\Users\Crazyblox\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Users\Crazyblox\node_modules\bluebird\js\release\promise.js:547:31)
at Promise._settlePromise (C:\Users\Crazyblox\node_modules\bluebird\js\release\promise.js:604:18)
at Promise._settlePromise0 (C:\Users\Crazyblox\node_modules\bluebird\js\release\promise.js:649:10)
at Promise._settlePromises (C:\Users\Crazyblox\node_modules\bluebird\js\release\promise.js:729:18)
at _drainQueueStep (C:\Users\Crazyblox\node_modules\bluebird\js\release\async.js:93:12)
at _drainQueue (C:\Users\Crazyblox\node_modules\bluebird\js\release\async.js:86:9)
at Async._drainQueues (C:\Users\Crazyblox\node_modules\bluebird\js\release\async.js:102:5)
at Async.drainQueues (C:\Users\Crazyblox\node_modules\bluebird\js\release\async.js:15:14)
at process.processImmediate (node:internal/timers:471:21)

Also having this issue.

Has been fixed. Please update to v4.14.0.


Add request body merger

I tried to add request body merger, because empty request body field is erased.
It works like as adding spec to this PR.

I thought POST request spec generator not merge but create new one.
But I tried to fix spec again, that fields were merged.
This PR is from my misunderstanding, so it would be close.
Thank you for reviewing!


Propagation finishes sucessfully without setting status to "2"

Propagation for channel "ch0" of [myToken] appears to have been successful as all resolutions down to level 10 contain data
http://openconnecto.me/ocp/ca/[myToken]/ch0/xz/10/0,15/7/0,1000/window/0,600/
But the propagate status is still "1" and has been that way for at least a week
http://openconnecto.me/ocp/ca/[myToken]/ch0/getPropagate/
When I try to manually set it to "2" to indicate success...
http://openconnecto.me/ocp/ca/[myToken]/ch0/setPropagate/2/
... I receive the following error
Invalid Value 2 for setPropagate

@kkutten1 This is resolved.


--output %(release_date)s gives "NA" on YouTube. Linux Mint 18.3, YTDL 2018.12.17.

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.12.17. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.12.17

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
youtube-dl -v 'https://www.youtube.com/watch?v=Kbyr0t8AIlE' --output "%(release_date)s"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=Kbyr0t8AIlE', u'--output', u'%(release_date)s']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.12.17
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-34-generic-x86_64-with-LinuxMint-18.3-sylvia
[debug] exe versions: ffmpeg 2.8.15-0ubuntu<IP_ADDRESS>, ffprobe 2.8.15-0ubuntu<IP_ADDRESS>
[debug] Proxy map: {}
[youtube] Kbyr0t8AIlE: Downloading webpage
[youtube] Kbyr0t8AIlE: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on u'https://r3---sn-aigzrn7k.googlevideo.com/videoplayback?expire=1546296197&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%2C298%2C299%2C302%2C303&lmt=1545154972286663&requiressl=yes&keepalive=yes&ip=<IP_ADDRESS>&clen=117984404&mt=1546274447&dur=316.533&mv=m&ms=au%2Crdu&mm=31%2C29&ipbits=0&c=WEB&mn=sn-aigzrn7k%2Csn-aigl6ned&gir=yes&initcwndbps=1266250&id=o-AJDMxmpMCyJCzvefeZvKrIhVO-rg9f9bygwYsWWxXjK2&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&ei=JEcqXKTEOoOE1wbTvbroDA&itag=299&beids=9466588&signature=C537B67E2E37E41CD712A61D559863AC6E29C87C.B36836C0A0E90A369B1380F1C2BB0DFE4D630008&fvip=3&mime=video%2Fmp4&pl=22&source=youtube&key=yt6&txp=5532432&ratebypass=yes'
[download] Destination: NA.f299
[download]   4.2% of 112.52MiB at  1.36MiB/s ETA 01:19^C
ERROR: Interrupted by user

Description of your issue, suggested solution and other information
When using youtube-dl on YouTube with the --output flag, the "%(release_date)s" tag results in only "NA" for all YouTube videos. Other tags, such as "%(title)s" still function normally. I am using Linux Mint 18.3 and youtube-dl version 2018.12.17, downloaded directly from your site, rather than my package manager. Thanks. :)

There is no release dates on youtube.


[FALSE-POSITIVE]

https://www.mercuriuscrypto.com/
Here is my domain, all the subwebsites and subdomain above it are beeing blocked by the same antivirus.
More Information
How did you discover your web site or domain was listed here?

My domain itself is beeing blocked by 5 antivirus, and they all are false-positive. I know that because I already made a scan in my domain by hostgator and got all clean. Also, 2 of the 5 antivirus already gave me a response confirming the false positive and removed my domain from their blacklist. Honestly, the problem is that anyone can inform me why this keeps happening, if my domain does not have any malware.

Have you requested removal from other sources?
Yes, well, you can check here in virus total, the remaining ones (now 3)
https://www.virustotal.com/gui/url/e32fcb0b6c0301bd85378af512ce592de2689e9c5e8f4fe64e08e5ef8bd26483/detection
Already got in contact with sophos and kasphesky, and both removed my website from blacklist and confirmed the false positive.
Additional context
Well, as I said, my domain is clean, and my provider confirmed that to me many times. Its a kind of false positive for sure, however I do not know why some antivirus are accusing my domain of this. If you could find out this for me I would be greateful, because, honestly, the most frustrating thing for me is not having any idea why my domain is being blocked if its all clean. I have a blog in wordpress, and I think its probably from there, but don't have a clue.
:exclamation:
We understand being listed on a Phishing Database like this can be frustrating and embarrassing for many web site owners. The first step is to remain calm. The second step is to rest assured one of our maintainers will address your issue as soon as possible. Please make sure you have provided as much information as possible to help speed up the process.
Send a Pull Request for faster removal
Users who understand github and creating Pull Requests can assist us with faster removals by sending a PR on the whitelist.me file
I have this virus total search:
https://www.virustotal.com/gui/url/e32fcb0b6c0301bd85378af512ce592de2689e9c5e8f4fe64e08e5ef8bd26483/detection

https://github.com/mitchellkrogza/Phishing.Database/blob/master/whitelist.me/whitelist.me
Im sorry, I didnt understand what I have to do on this whitelist.me, If you explain it to me shortly I can gladly do it. Thanks

As of now, I can't find any occurrences of the subject of this issue.
Closing.
Stay safe and healthy.
Nissar

@Mercurius123221 as of now, I still can't find any occurrence of the subject of this issue in our filesystem.
$ fgrep -R --exclude-dir .git mercuriuscrypto.com -R .
$ fgrep -R --exclude-dir .git www.mercuriuscrypto.com -R .

Closing.
Stay safe and healthy.
Nissar

Hello, I just got back to virus total team. Because my website is still
being claimed to be blocked by phishing database. So virus total send to me
that:
[image: image.png]
Could you please warn them thats all good with the domain by you?
Thanks
On Sun, Jul 12, 2020 at 12:32 PM Nissar Chababy<EMAIL_ADDRESS>wrote:

Closed #94 https://github.com/mitchellkrogza/Phishing.Database/issues/94
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mitchellkrogza/Phishing.Database/issues/94#event-3537201894,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQG5NPL4J7PSAVJ4J436TMLR3HJSDANCNFSM4OUSZ3DA
.

@Mercurius123221 I can't see the image you uploaded per email. Please repost it.

Sorry!
Here it is. At the attatchments
On Tue, Jul 14, 2020 at 4:18 PM Nissar Chababy<EMAIL_ADDRESS>wrote:

@Mercurius123221 https://github.com/Mercurius123221 I can't see the
image you uploaded per email. Please repost it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mitchellkrogza/Phishing.Database/issues/94#issuecomment-658364695,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQG5NPIZKT4NRUBVLMIMXNDR3SVR3ANCNFSM4OUSZ3DA
.

Still no attachment.... You would probably have to login to GH to attach the image...

Ok!
Just submitted new issue then. Here is the link:
https://github.com/mitchellkrogza/Phishing.Database/issues/101
With the image there!
Waiting for your response, thank you
On Thu, Jul 16, 2020 at 6:12 PM spirillen<EMAIL_ADDRESS>wrote:

Still no attachment.... You would probably have to login to GH to attach
the image...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mitchellkrogza/Phishing.Database/issues/94#issuecomment-659675570,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQG5NPMW6GYI2OBRHVZ4PWDR35UK3ANCNFSM4OUSZ3DA
.


In-app Chat

Description
According to the Database, we need an In-app chat, the dynamic is simply, only the users can start a chat with the coworkers owners.

[x] Need to create a chats page
[x]  Add Chat button in spaces

[x] Added the space owner information in space details

[ ] Added bell to show notifications into the main header

[ ] Added read properties read_for_from_user and read_for_to_user to the chat entity

Any others features are welcome.

@chakrihacker I have a request for you 😃 When you have time, please add properties to chat entity to make chat read by users read_for_from_user and read_for_to_user

Added bell icon

I added a function and trigger on Supabase
Function

begin
  insert into public.notification(type, to_user)
  values('message', (select to_user from public.chats where id = new.chat_id));

  return new;
end;

Trigger

Created function and trigger to handle the notifications of the schedules
Function

begin
  insert into public.notification(type, to_user, meta)
  values(
    'schedule',
    (select space.userid
from public.spaces space
inner join public.users on public.users.id = space.userid
where space.id = new.space_id),
    format('{"space_id":"%s", "schedule_id":"%s"}', new.space_id, new.id)::jsonb
  );

  return new;
end;

Trigger

The schedule notification uses a combination of schedule_id query parameter and in-code highlight, when the user clicked any notification, is redirected to the profile and the app automatic scroll down to the schedule and add a background color


Install issue

After installing @fratton's fork things aren't working. (I can't report bugs there because he hasn't enabled that, sorry!) So...

@fratton please enable bug reporting unless you and @milo526 would prefer to consolidate here, of course. Not sure what you prefer.
@fratton fork installs into "@fratton" instead of what I think is more standard homebridge-tuna-web or something similar.
I can no longer control my Tuya accessories with HomeKit. I will try installing an older version of the plugin and see if things work and update the bug.

If you would like these broken into separate bugs, of course let me know.

O.K. I switched back to @milo526's fork and latest is now working for me. However, the install gives minor errors:
/homebridge # npm install<EMAIL_ADDRESS>+<EMAIL_ADDRESS>added 4 packages from 7 contributors, removed 47 packages, updated 1 package and audited 583 packages in 27.095s

33 packages are looking for funding
  run `npm fund` for details

found 2 vulnerabilities (1 low, 1 moderate)
  run `npm audit fix` to fix them, or `npm audit` for details
/homebridge # npm audit
npm ERR! code ELOCKVERIFY
npm ERR! Errors were found in your package-lock.json, run  npm install  to fix them.
npm ERR!     Invalid: lock file's<EMAIL_ADDRESS>does not satisfy<EMAIL_ADDRESS>
npm ERR!
It looks like you have a package-lock.json file (why? this shouldn't be necessary) and said file seems not to be updating when running npm install which is also strange.
The latest version of this library seems not to have any security issues if I run npm audit locally`
Could you please see if moving your lock-file and having npm generate a new one helps to solve this problem?

I’d be happy to if I could find it.

I think the file you are looking for is in /homebridge/package-lock.json (are you using homebridge-config-ui-x?)

That file doesn't exist.
yes, I'm using homebridge-config-ui-x.

So I just tried it again and got the same thing.
/homebridge # npm install<EMAIL_ADDRESS>+<EMAIL_ADDRESS>updated 1 package and audited 635 packages in 23.307s

31 packages are looking for funding
  run `npm fund` for details

found 2 vulnerabilities (1 low, 1 moderate)
  run `npm audit fix` to fix them, or `npm audit` for details
/homebridge #
/homebridge # npm audit
npm ERR! code EAUDITNOLOCK
npm ERR! audit Neither npm-shrinkwrap.json nor package-lock.json found: Cannot audit a project without a lockfile
npm ERR! audit Try creating one first with: npm i --package-lock-only

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-06-20T22_43_13_973Z-debug.log
/homebridge #

So i am kind of surprised by the fact that nobody else received this error (no issues mentioned on gitlab, reddit or directly to me). I also cannot reproduce this with a clean install.
For now I'll close this issue, if you can reproduce this on a clean install; please mention all steps you took to setup your homebridge + this plugin.

FYI https://github.com/oznu/homebridge-config-ui-x/issues/767


Using License3J on Java 8 for linux

Hello, I can only use java 8, but I still want to use your License3j.
Should I try the 2.0.0 release ? Or it is already outdated?

It is not outdated in the sense that it can be used. Just be prepared that the license handling and license format is totally different in later versions. Whenever you migrate to Java 11 and the new version of License3j you will have to replace all the issued license files and rewrite the license handling API. You can also use the old version of License3j and Java 9 or later. This is also a possibility. It is up to you.
The fact that there is a newer version does not make the old version any worse. It is just as good as it was before.

Btw. License3j 2.0.0 also requires Java 10 as a minimum.

Ok, I should be on the License3j 1.0.8 version.
Is there a major difference between 1.0.8 and 2.0.0 version?

I tried the 2.0.0, it didn't tell me any problems

I want to run it on linux, the license3j.bat file is for window right? So is here any other way?

You can try to use license3j.sh. That may work under Linux.

license3j.bat encode --license-file=license-plain.
txt --keyring-file="c:\Documents and Settings\verhas\Application Data\gnupg\secr
ing.gpg" --key=GrooWikiDemoKey --password=******** --output=license.out
this is the command you ask us to do for generate the license file with window, I don't think license3j.sh do the same as license3j.bat do for linux

Or maybe shoud I use windows to generate license file and linux for the validation of the license file?

"This time you have to start the command line processor included in the license3j JAR file."
When you said this just before telling us to do the license3j.bat encode:

what is the command line processor?
And where can we find the license3j JAR file? (tried mvn package but didn t work)

You were right, it seem, I can't use java 8 for License3j 2.0.0 to generate license.
May I ask you to upload the version 1.0.8 of License3j?

java -cp target/license3j-2.0.0-JVM8.jar com.javax0.license3j.License3j encode --license-file=license-plain.txt --keyring-file=secring.gpg --key=licensetest --password=password --output=license.out
I get this after the command above, what is the meaning of this error? I have org.bouncycastle version 1.57 in your pom.xml as dependency from your 2.0.0 release git
Exception in thread "main" java.lang.NoClassDefFoundError: org/bouncycastle/openpgp/PGPException
at com.javax0.license3j.License3j.encode(License3j.java:97)
at com.javax0.license3j.License3j.main(License3j.java:86)
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.openpgp.PGPException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

You can download all the published versions from
https://mvnrepository.com/artifact/com.verhas/license3j

The License3j version 2.0.0 has a dependency on the BouncyCastle library, as you can see in the POM file. This library needs to be there on the classpath when the application runs and not only during compile time.
Your command line has only the license3j JAR file on the classpath.

I tried this one alse, but still I get the same error:
java -cp target/license3j-2.0.0-JVM8.jar:/home/ivan/.m2/repository/org/bouncycastle/bcpg-jdk15on/1.57/bcpg-jdk15on-1.57-sources.jar:/home/ivan/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.57/bcprov-jdk15on-1.57-sources.jar:/home/ivan/.m2/repository/org/mockito/mockito-all/1.10.17/mockito-all-1.10.17-sources.jar:/home/ivan/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.2.0/junit-jupiter-api-5.2.0-sources.jar:/home/ivan/.m2/repository/org/apiguardian/apiguardian-api/1.0.0/apiguardian-api-1.0.0-sources.jar:/home/ivan/.m2/repository/org/opentest4j/opentest4j/1.1.0/opentest4j-1.1.0-sources.jar:/home/ivan/.m2/repository/org/junit/platform/junit-platform-commons/1.2.0/junit-platform-commons-1.2.0-sources.jar:/home/ivan/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.2.0/junit-jupiter-engine-5.2.0-sources.jar:/home/ivan/.m2/repository/org/junit/platform/junit-platform-engine/1.2.0/junit-platform-engine-1.2.0-sources.jar com.javax0.license3j.License3j encode --license-file=license-plain.txt --keyring-file=secring.gpg --key=licensetest --password=password --output=license.out
Exception in thread "main" java.lang.NoClassDefFoundError: org/bouncycastle/jce/provider/BouncyCastleProvider
at com.javax0.license3j.License3j.encode(License3j.java:97)
at com.javax0.license3j.License3j.main(License3j.java:86)
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.jce.provider.BouncyCastleProvider
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)

There is no surprise. You will get the same error again and again while you do not have the BouncyCastle library on the classpath. You have a lot of unused libraries on the classpath, but the BouncyCastle library is not on the classpath.


fix: list button style

:tada: This PR is included in version 1.138.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


linux: use io_uring for network i/o

This is an invitation to brainstorm. :-)
There is currently a mismatch between io_uring's and libuv's I/O model that stops libuv from achieving maximal performance if it were to use io_uring for network i/o, particularly when it comes to receiving incoming connections and packets.
https://github.com/axboe/liburing/wiki/io_uring-and-networking-in-2023 describes best practices that can be summarized as:

keep multiple i/o requests enqueued (mismatch: libuv's "firehose" approach to incoming connections/data)

let io_uring manage buffers (mismatch: libuv punts memory management to the user)

I've been thinking about what needs to change and this is what I came up with so far:

add request-based apis for reading data and accepting connections, like int uv_stream_read(uv_read_t* req, uv_stream_t* handle, uv_buf_t* bufs, size_t nbufs, unsigned flags, uv_stream_read_cb cb) where uv_stream_read_cb is void (*)(uv_read_t* req, ssize_t nread, uv_buf_t* bufs)

add a memory pool api that tells libuv it can allocate this much memory for buffers. Strawman proposal: uv_loop_configure(loop, UV_LOOP_SET_BUFFER_POOL_SIZE, 8<<20) with 0 meaning "pick a suitable default"

introduce a flag that tells uv_stream_read() to ignore bufs and interpret nbufs as "bytes to read into buffer pool" (kind of ugly, suggestions welcome)

introduce a new int uv_release_buffers(loop, bufs, nbufs) (maybe s/loop/handle/?) that tells libuv it's okay to reuse those buffer pool slices again. Alternatively: reclaim the memory automatically when uv_stream_read_cb returns but that means users may have to copy, or that they hold on longer to the buffers than is needed (inefficient if they queue up new requests in the callback)

Libuv-on-Windows is internally already request-based so should be relatively easy to adapt.
Suggestions for improvements very welcome!

From a library user aspect:

Request-based API is much easier to be integrated than Firehose style. We used to add lots of hacks to get the correct accept loop because we can't stop inside connection callback(the read is stoppable so the problem is not so bad).

For buffer management, it became complicated, If the user callback deserializes the buffer into some struct, and the buffer is supposed to be reused, the user has to be careful don't point to the buffer (as you said: have to copy, or that they hold on longer to the buffers than is needed). Maybe adding buffer management API is not worth the effort.

I've been thinking about what needs to change and this is what I came up with so far:

add request-based apis for reading data and accepting connections, like int uv_stream_read(uv_read_t* req, uv_stream_t* handle, uv_buf_t* bufs, size_t nbufs, unsigned flags, uv_stream_read_cb cb) where uv_stream_read_cb is void (*)(uv_read_t* req, ssize_t nread, uv_buf_t* bufs)

:+1: to this, I was bitten by no having this kind of api when I was experimenting with iouring in the past. Also, something that moves https://github.com/libuv/leps/pull/2 forward is a plus.

add a memory pool api that tells libuv it can allocate this much memory for buffers. Strawman proposal: uv_loop_configure(loop, UV_LOOP_SET_BUFFER_POOL_SIZE, 8<<20) with 0 meaning "pick a suitable default"

How would this work exactly? IIUC, IORING_OP_PROVIDE_BUFFERS needs also to define the number of buffers and their size. Would we need to define at least one of those or would that be an implementation detail libuv takes care of? In the same vein, I see it's recommended in some cases to register multiple sets of buffers. Would also this be an implementation detail?

introduce a flag that tells uv_stream_read() to ignore bufs and interpret nbufs as "bytes to read into buffer pool" (kind of ugly, suggestions welcome)

lgtm.

introduce a new int uv_release_buffers(loop, bufs, nbufs) (maybe s/loop/handle/?) that tells libuv it's okay to reuse those buffer pool slices again. Alternatively: reclaim the memory automatically when uv_stream_read_cb returns but that means users may have to copy, or that they hold on longer to the buffers than is needed (inefficient if they queue up new requests in the callback)

Why the s/loop/handle/?
I'm torn on this one, while I like the flexibility that having an API like that, it seems to me it would make it much more difficult for users to have it right. Could we have a flag UV_STREAM_NOT_RECLAIM_MEMORY or smthing like that, that would allow us to have both options available?

Yes, it would be entirely up to libuv to decide how to chop up the allotment; I don't want too many io_uring eccentricities sneaking into the API because it also needs to work meaningfully on other platforms.
That said, I'm leaning towards shelving buffer management for now and just implement the simple thing first. The flags argument lets us add it later as an opt-in.

Cool stuff. I just wanted to mention two other projects which are in the same area, abstracting IO over multiple platforms like Linux with io_uring, Windows IOCP and Macos. The first is libxev by Mitchell Hashimoto (https://github.com/mitchellh/libxev) which is a library for doing async IO, adopting the io_uring style (perhaps good to study for how it does work on other platforms?). The second is TigerBeetle which is a database but it has the IO fairly neatly in a directory (https://github.com/tigerbeetle/tigerbeetle/tree/main/src/io) so it's quite easy to read. Both are written in Zig, though libxev exposes a C API. Perhaps one can draw some inspiration from their work :)

I opt for keeping the firehose approach incoming connections/data. io_uring has a  handy feature called multshot(https://man.archlinux.org/man/io_uring.7.en#IORING_CQE_F_MORE), applicable to accept(43), read(0). Multshot can be depicted as one submission(SQE), multiple completeness(CQE). This feature closely matches the handle-based API in libuv.

If we switch reading from handle-based to request-based, the first dilemma comes to me is that how we submit read operation to io_uring, one by one (io_uring_enter(ring_fd, 1/*to_submit*/, ...)) or by batch(io_uring_enter(ring_fd, n/*to_submit*/, ...)) ? The first way means libuv will submit a read operation to io_uring every time the user issues a read request, which is good for latency but more syscalls are incurred. While the latter one means libuv will collect read requests and submit them all in a later time, which has opposite pros/cons.

If we switch reading from handle-based to request-based, the first dilemma comes to me is that how we submit read operation to io_uring, one by one (io_uring_enter(ring_fd, 1/*to_submit*/, ...)) or by batch(io_uring_enter(ring_fd, n/*to_submit*/, ...)) ? The first way means libuv will submit a read operation to io_uring every time the user issues a read request, which is good for latency but more syscalls are incurred. While the latter one means libuv will collect read requests and submit them all in a later time, which has opposite pros/cons.

Ignore this, pls. I just spot that libuv makes use of IORING_SETUP_SQPOLL, which makes us exempted from submiting operations(SQE)


Using catalog.getNames return an empty set

First i want to say a big thanks for spring-cloud-function !
My issue :
When using catalog.getNames, it returns an empty set.
I've defined six functions as classes :

one as a Supplier
five as Function (as Function<String, String> or Function<Flux, Flux>)

In my @SpringBootApplication i define :
public static void main(String[] args) {
	ConfigurableApplicationContext context =
			SpringApplication.run(FunctionApplication.class, args);
	String[] singletonNames = context.getBeanDefinitionNames();
	for (String singleton : singletonNames) {
		System.out.println(singleton);
	}
	FunctionCatalog catalog = (FunctionCatalog) context.getBean("functionCatalog");
	System.out.println(catalog.getNames(Supplier.class));
	System.out.println(catalog.getNames(Consumer.class));
	System.out.println(catalog.getNames(Function.class));
	System.out.println(catalog.size());
	context.close();
}

This print :
[listOfAllFunctions]
[]
[]
6
Indeed i have a function whose name is ListOfAllFunctions and a total of 6 functions but catalog.getNames(Function.class) return an empty Set.
Am i using getNames wrongly ?
Thanks a lot.
(i do this because i want to list available Functions in my "Service Core" which calls Functions)

Strange indeed. I just tried and everything works as expected (see the full code below).
@SpringBootApplication
public class Issue275App {

	public static void main(String[] args) {
		ApplicationContext context = SpringApplication.run(Issue275App.class);
		FunctionCatalog catalog = context.getBean(FunctionCatalog.class);
		System.out.println(catalog.getNames(Function.class));
		System.out.println(catalog.getNames(Supplier.class));
		System.out.println(catalog.getNames(Consumer.class));
	}

	@Bean
	public Function<String, String> echo() {
		return v -> v;
	}

	@Bean
	public Function<Flux<Long>, Flux<Long>> foo() {
		return v -> v;
	}

        @Bean
	public Consumer<Flux<String>> consume() {
		return v -> System.out.println(v);
	}
}

. . . and output:
[foo, consume, echo]
[]
[]

You can see that consume returned as Function since it is effectively wrapped into Function<Flux, Mono>`, but that is another discussion. The point is hat all functions that are defined are available in catalog.
In any event, would you be able to provide full sample that reproduces it?

Thanks !
Yes you can get it here : https://github.com/neokeld/function-example

Ok that was a bad import :
import org.apache.el.lang.FunctionMapperImpl.Function;
instead of
import java.util.function.Function;
There is no bug.
Thanks a lot for your help !

I guess it is no longer an issue. Closing


Optimize bytes_queue

Hi there, I try to optimize bytes_queue in two ways:

The length is encoded in varint format(the assertion on capacity will fail so I commented them out).
When we insert an entry before the head of the queue, and the size of this entry exactly equals to unsued bytes, there is no need to reserve bytes.

And below are the outputs:
# Mine
2020/03/12 14:06:01 Allocated new queue in 133ns; Capacity: 88
2020/03/12 14:06:01 Allocated new queue in 211.388µs; Capacity: 585000
2020/03/12 14:06:01 Allocated new queue in 595.952µs; Capacity: 1170000
PASS

# Yours
2020/03/12 14:06:56 Allocated new queue in 156ns; Capacity: 94
2020/03/12 14:06:56 Allocated new queue in 719.26µs; Capacity: 585000
2020/03/12 14:06:56 Allocated new queue in 566.888µs; Capacity: 1170000
PASS

PS: It can save some bytes and avoid re-allocation of queue at some time

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@75a65a8). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #207   +/-   ##
=========================================
  Coverage          ?   89.18%
=========================================
  Files             ?       15
  Lines             ?      703
  Branches          ?        0
=========================================
  Hits              ?      627
  Misses            ?       65
  Partials          ?       11

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75a65a8...410d43d. Read the comment docs.

@janisz Hi, here's the benchmark result:
# Mine
2020/03/12 19:02:50 Allocated new queue in 180ns; Capacity: 88
2020/03/12 19:02:50 Allocated new queue in 226.702µs; Capacity: 585000
2020/03/12 19:02:50 Allocated new queue in 389.079µs; Capacity: 1170000
goos: darwin
goarch: amd64
pkg: github.com/allegro/bigcache/v2
BenchmarkWriteToCacheWith1Shard-4                            	 5192714	      1026 ns/op	     596 B/op	       3 allocs/op
BenchmarkWriteToLimitedCacheWithSmallInitSizeAnd1Shard-4     	 8772144	       508 ns/op	     104 B/op	       4 allocs/op
BenchmarkWriteToUnlimitedCacheWithSmallInitSizeAnd1Shard-4   	 1902460	      5424 ns/op	    3623 B/op	       2 allocs/op
BenchmarkWriteToCache/1-shards-4                             	 3431140	      1486 ns/op	     620 B/op	       3 allocs/op
BenchmarkWriteToCache/512-shards-4                           	 9916384	      1135 ns/op	     599 B/op	       3 allocs/op
BenchmarkWriteToCache/1024-shards-4                          	 7211010	       999 ns/op	     618 B/op	       3 allocs/op
BenchmarkWriteToCache/8192-shards-4                          	 6708818	      1065 ns/op	     602 B/op	       3 allocs/op
BenchmarkReadFromCache/1-shards-4                            	13963962	      2549 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCache/512-shards-4                          	 9840069	       828 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCache/1024-shards-4                         	 9384037	       923 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCache/8192-shards-4                         	10724266	      1131 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/1-shards-4                    	10171226	      1048 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/512-shards-4                  	 8341252	       888 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/1024-shards-4                 	10288347	      1135 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/8192-shards-4                 	 8065857	       827 ns/op	     271 B/op	       2 allocs/op
BenchmarkIterateOverCache/512-shards-4                       	24373030	       228 ns/op	      20 B/op	       2 allocs/op
BenchmarkIterateOverCache/1024-shards-4                      	22172804	       214 ns/op	      20 B/op	       2 allocs/op
BenchmarkIterateOverCache/8192-shards-4                      	22551939	       220 ns/op	      20 B/op	       2 allocs/op
BenchmarkWriteToCacheWith1024ShardsAndSmallShardInitSize-4   	 5228383	      1452 ns/op	    1108 B/op	       3 allocs/op
BenchmarkReadFromCacheNonExistentKeys/1-shards-4             	36336122	       137 ns/op	       7 B/op	       0 allocs/op
BenchmarkReadFromCacheNonExistentKeys/512-shards-4           	38118368	       141 ns/op	       7 B/op	       0 allocs/op
BenchmarkReadFromCacheNonExistentKeys/1024-shards-4          	36579570	       140 ns/op	       7 B/op	       0 allocs/op
BenchmarkReadFromCacheNonExistentKeys/8192-shards-4          	36496588	       149 ns/op	       7 B/op	       0 allocs/op
BenchmarkFnvHashSum64-4                                      	460839229	        10.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkFnvHashStdLibSum64-4                                	90994177	        67.8 ns/op	      16 B/op	       2 allocs/op
PASS
ok  	github.com/allegro/bigcache/v2	555.615s
PASS
ok  	github.com/allegro/bigcache/v2/queue	0.012s
2020/03/12 19:12:05 Entry not found
2020/03/12 19:12:05 stored "testkey" in cache.
2020/03/12 19:12:05 empty request.
2020/03/12 19:12:05 stored "putKey" in cache.
2020/03/12 19:12:05 Entry not found
2020/03/12 19:12:05 test read error
2020/03/12 19:12:05 invalidDeleteKey not found.
2020/03/12 19:12:05  not found.
2020/03/12 19:12:05 empty request.
2020/03/12 19:12:05 entry is bigger than max shard size
PASS

# Yours
2020/03/12 18:48:16 Allocated new queue in 264.583µs; Capacity: 585000
2020/03/12 18:48:16 Allocated new queue in 182ns; Capacity: 94
2020/03/12 18:48:16 Allocated new queue in 478.836µs; Capacity: 1170000
goos: darwin
goarch: amd64
pkg: github.com/allegro/bigcache/v2
BenchmarkWriteToCacheWith1Shard-4                            	 4990784	      1342 ns/op	     598 B/op	       3 allocs/op
BenchmarkWriteToLimitedCacheWithSmallInitSizeAnd1Shard-4     	 8682642	       566 ns/op	      40 B/op	       3 allocs/op
BenchmarkWriteToUnlimitedCacheWithSmallInitSizeAnd1Shard-4   	 3486128	      8048 ns/op	    3953 B/op	       2 allocs/op
BenchmarkWriteToCache/1-shards-4                             	 3656758	      1755 ns/op	     616 B/op	       3 allocs/op
BenchmarkWriteToCache/512-shards-4                           	 5487960	      1203 ns/op	     595 B/op	       3 allocs/op
BenchmarkWriteToCache/1024-shards-4                          	 7292827	       952 ns/op	     617 B/op	       3 allocs/op
BenchmarkWriteToCache/8192-shards-4                          	 7152934	       986 ns/op	     631 B/op	       3 allocs/op
BenchmarkReadFromCache/1-shards-4                            	13202287	      2494 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCache/512-shards-4                          	 9122005	      1506 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCache/1024-shards-4                         	10188386	       964 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCache/8192-shards-4                         	10101868	      1393 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/1-shards-4                    	 8598147	       830 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/512-shards-4                  	 9947847	       873 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/1024-shards-4                 	 9378534	       900 ns/op	     271 B/op	       2 allocs/op
BenchmarkReadFromCacheWithInfo/8192-shards-4                 	 8794632	       953 ns/op	     271 B/op	       2 allocs/op
BenchmarkIterateOverCache/512-shards-4                       	21350042	       216 ns/op	      20 B/op	       2 allocs/op
BenchmarkIterateOverCache/1024-shards-4                      	20962723	       224 ns/op	      20 B/op	       2 allocs/op
BenchmarkIterateOverCache/8192-shards-4                      	21533272	       205 ns/op	      20 B/op	       2 allocs/op
BenchmarkWriteToCacheWith1024ShardsAndSmallShardInitSize-4   	 4855218	      1543 ns/op	    1190 B/op	       3 allocs/op
BenchmarkReadFromCacheNonExistentKeys/1-shards-4             	30844810	       150 ns/op	       7 B/op	       0 allocs/op
BenchmarkReadFromCacheNonExistentKeys/512-shards-4           	33429462	       139 ns/op	       7 B/op	       0 allocs/op
BenchmarkReadFromCacheNonExistentKeys/1024-shards-4          	32085747	       149 ns/op	       7 B/op	       0 allocs/op
BenchmarkReadFromCacheNonExistentKeys/8192-shards-4          	37587307	       141 ns/op	       7 B/op	       0 allocs/op
BenchmarkFnvHashSum64-4                                      	519146876	         9.40 ns/op	       0 B/op	       0 allocs/op
BenchmarkFnvHashStdLibSum64-4                                	70484605	        64.3 ns/op	      16 B/op	       2 allocs/op
PASS
ok  	github.com/allegro/bigcache/v2	556.285s
PASS
ok  	github.com/allegro/bigcache/v2/queue	0.010s
2020/03/12 18:57:32 Entry not found
2020/03/12 18:57:32 stored "testkey" in cache.
2020/03/12 18:57:32 empty request.
2020/03/12 18:57:32 stored "putKey" in cache.
2020/03/12 18:57:32 Entry not found
2020/03/12 18:57:32 empty request.
2020/03/12 18:57:32 invalidDeleteKey not found.
2020/03/12 18:57:32  not found.
2020/03/12 18:57:32 test read error
2020/03/12 18:57:32 entry is bigger than max shard size
PASS

I'm sorry that I'm not familiar with the go benchmark test, so I can not add a new benchmark for the encoding part. But according to the above result, I think using new uvarint encoding is not insufficient.
And It can really reduce the consumption of memory.

@mxplusb I would not like to lose test coverage too, sir.
But I think the problem is the old assertions about capacity rely on implementation detail too much, like this test.

Why does the build fail? My pr should have nothing to do with data race I think.

Greate! I would also like to help you guys with this issue


Optionally preserve keys in Field nested values

Per discussion on IRC.
Includes tests. That work. I tested them. Really. :D

Coverage remained the same at 100.0% when pulling b3bba7146597fc2c9fcd9de010aea987a5144bce on MichaelGooden:feature/nested-values-preserve-keys into 94ebc0b9b08cafd4ebd60b661a0eab5b85c4bc3f on DASPRiD:master.

Coverage remained the same at 100.0% when pulling e528fe14a49462f1fa3e8e66090c05045dfa750f on MichaelGooden:feature/nested-values-preserve-keys into 94ebc0b9b08cafd4ebd60b661a0eab5b85c4bc3f on DASPRiD:master.

Coverage remained the same at 100.0% when pulling 79c43a7ee5179d503cd2a401be4c67f65d2c36c2 on MichaelGooden:feature/nested-values-preserve-keys into 94ebc0b9b08cafd4ebd60b661a0eab5b85c4bc3f on DASPRiD:master.


Support for MuMuQA a multi-hop QA task over image-caption pairs

Support for MuMuQA a benchmark QA task that involves multi-hop question answering over image-caption pairs.
Based on the paper MuMuQA: Multimedia Multi-Hop News Question Answering via
Cross-Media Knowledge Extraction and Grounding Reddy, R. G, et.al
Be able to reproduce the baselines

Image feature extraction and representation is done using:
https://github.com/rbgirshick/py-faster-rcnn
https://github.com/microsoft/Oscar

@bhavani105  @gangiswag Is there any status update for this one ? If yes, could you please put it here ?


Use ProcessorTop templates in non osflow examples

Currently, the non osflow examples (radiant, quartus) do not use a ProcessorTop or SystemTop template. Quartus is trying to use neorv32_test_setup.vhd (which was renamed and does not exist) and radiant is instantiating the NEORV32 top directly.
It would be nice to update these examples for using neorv32_ProcessorTop_Test and neorv32_ProcessorTop_UP5KDemo (or MinimalBoot).
Vivado examples use neorv32_ProcessorTop_Test already.

While I acknowledge that Minimal vs MinimalBoot might be confusing, the main idea is that Minimal has the bootloader disabled and no other output than the PWM. Conversely, in MinimalBoot the usage of the bootloader is optional, as it is connecting the UART or GPIO. Therefore, I think that "indirectboot" and "directboot" are not descriptive enough. The purpose of MinimalBoot is to be used for both indirect and direct boot. It is correct that Minimal is direct only.
Other than that, I think that Tiny, Small, Medium, etc. might be as good as any other naming you pick.

maybe we could add _wishbone and _axilite to highlight connectivity if a certain setup provides a bus interface

We are already doing that: https://github.com/stnolting/neorv32/blob/master/rtl/templates/system/neorv32_SystemTop_axi4lite.vhd. So, yes.

I was just thinking out loud 😅

Other than that, I think that Tiny, Small, Medium, etc. might be as good as any other naming you pick.

I am still not sure about this...
Anyway, I think it is a good idea to change the current Intel and Xilinx setups so they use one of the template instances. 👍

Fixed ✔

Other than that, I think that Tiny, Small, Medium, etc. might be as good as any other naming you pick.

I am still struggling with this 😅 but that might be stuff for a different issue.
I think this can be closed now.


Synthesis failed for ServiceNetworking

Hello! Autosynth couldn't regenerate ServiceNetworking. :broken_heart:
Here's the output from running synth.py:
service_networking/v1/model/billing_destination.ex.
Writing CancelOperationRequest to clients/service_networking/lib/google_api/service_networking/v1/model/cancel_operation_request.ex.
Writing Connection to clients/service_networking/lib/google_api/service_networking/v1/model/connection.ex.
Writing ConsumerProject to clients/service_networking/lib/google_api/service_networking/v1/model/consumer_project.ex.
Writing Context to clients/service_networking/lib/google_api/service_networking/v1/model/context.ex.
Writing ContextRule to clients/service_networking/lib/google_api/service_networking/v1/model/context_rule.ex.
Writing Control to clients/service_networking/lib/google_api/service_networking/v1/model/control.ex.
Writing CustomError to clients/service_networking/lib/google_api/service_networking/v1/model/custom_error.ex.
Writing CustomErrorRule to clients/service_networking/lib/google_api/service_networking/v1/model/custom_error_rule.ex.
Writing CustomHttpPattern to clients/service_networking/lib/google_api/service_networking/v1/model/custom_http_pattern.ex.
Writing DisableVpcServiceControlsRequest to clients/service_networking/lib/google_api/service_networking/v1/model/disable_vpc_service_controls_request.ex.
Writing DnsRecordSet to clients/service_networking/lib/google_api/service_networking/v1/model/dns_record_set.ex.
Writing DnsZone to clients/service_networking/lib/google_api/service_networking/v1/model/dns_zone.ex.
Writing Documentation to clients/service_networking/lib/google_api/service_networking/v1/model/documentation.ex.
Writing DocumentationRule to clients/service_networking/lib/google_api/service_networking/v1/model/documentation_rule.ex.
Writing Empty to clients/service_networking/lib/google_api/service_networking/v1/model/empty.ex.
Writing EnableVpcServiceControlsRequest to clients/service_networking/lib/google_api/service_networking/v1/model/enable_vpc_service_controls_request.ex.
Writing Endpoint to clients/service_networking/lib/google_api/service_networking/v1/model/endpoint.ex.
Writing Enum to clients/service_networking/lib/google_api/service_networking/v1/model/enum.ex.
Writing EnumValue to clients/service_networking/lib/google_api/service_networking/v1/model/enum_value.ex.
Writing Field to clients/service_networking/lib/google_api/service_networking/v1/model/field.ex.
Writing GoogleCloudServicenetworkingV1betaSubnetwork to clients/service_networking/lib/google_api/service_networking/v1/model/google_cloud_servicenetworking_v1beta_subnetwork.ex.
Writing Http to clients/service_networking/lib/google_api/service_networking/v1/model/http.ex.
Writing HttpRule to clients/service_networking/lib/google_api/service_networking/v1/model/http_rule.ex.
Writing JwtLocation to clients/service_networking/lib/google_api/service_networking/v1/model/jwt_location.ex.
Writing LabelDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/label_descriptor.ex.
Writing ListConnectionsResponse to clients/service_networking/lib/google_api/service_networking/v1/model/list_connections_response.ex.
Writing ListOperationsResponse to clients/service_networking/lib/google_api/service_networking/v1/model/list_operations_response.ex.
Writing LogDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/log_descriptor.ex.
Writing Logging to clients/service_networking/lib/google_api/service_networking/v1/model/logging.ex.
Writing LoggingDestination to clients/service_networking/lib/google_api/service_networking/v1/model/logging_destination.ex.
Writing Method to clients/service_networking/lib/google_api/service_networking/v1/model/method.ex.
Writing MetricDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/metric_descriptor.ex.
Writing MetricDescriptorMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/metric_descriptor_metadata.ex.
Writing MetricRule to clients/service_networking/lib/google_api/service_networking/v1/model/metric_rule.ex.
Writing Mixin to clients/service_networking/lib/google_api/service_networking/v1/model/mixin.ex.
Writing MonitoredResourceDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/monitored_resource_descriptor.ex.
Writing Monitoring to clients/service_networking/lib/google_api/service_networking/v1/model/monitoring.ex.
Writing MonitoringDestination to clients/service_networking/lib/google_api/service_networking/v1/model/monitoring_destination.ex.
Writing OAuthRequirements to clients/service_networking/lib/google_api/service_networking/v1/model/o_auth_requirements.ex.
Writing Operation to clients/service_networking/lib/google_api/service_networking/v1/model/operation.ex.
Writing Option to clients/service_networking/lib/google_api/service_networking/v1/model/option.ex.
Writing Page to clients/service_networking/lib/google_api/service_networking/v1/model/page.ex.
Writing PolicyBinding to clients/service_networking/lib/google_api/service_networking/v1/model/policy_binding.ex.
Writing Quota to clients/service_networking/lib/google_api/service_networking/v1/model/quota.ex.
Writing QuotaLimit to clients/service_networking/lib/google_api/service_networking/v1/model/quota_limit.ex.
Writing Range to clients/service_networking/lib/google_api/service_networking/v1/model/range.ex.
Writing RangeReservation to clients/service_networking/lib/google_api/service_networking/v1/model/range_reservation.ex.
Writing RemoveDnsRecordSetMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_metadata.ex.
Writing RemoveDnsRecordSetRequest to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_request.ex.
Writing RemoveDnsRecordSetResponse to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_response.ex.
Writing RemoveDnsZoneMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_metadata.ex.
Writing RemoveDnsZoneRequest to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_request.ex.
Writing RemoveDnsZoneResponse to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_response.ex.
Writing Route to clients/service_networking/lib/google_api/service_networking/v1/model/route.ex.
Writing SearchRangeRequest to clients/service_networking/lib/google_api/service_networking/v1/model/search_range_request.ex.
Writing Service to clients/service_networking/lib/google_api/service_networking/v1/model/service.ex.
Writing ServiceIdentity to clients/service_networking/lib/google_api/service_networking/v1/model/service_identity.ex.
Writing SourceContext to clients/service_networking/lib/google_api/service_networking/v1/model/source_context.ex.
Writing SourceInfo to clients/service_networking/lib/google_api/service_networking/v1/model/source_info.ex.
Writing Status to clients/service_networking/lib/google_api/service_networking/v1/model/status.ex.
Writing Subnetwork to clients/service_networking/lib/google_api/service_networking/v1/model/subnetwork.ex.
Writing SystemParameter to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameter.ex.
Writing SystemParameterRule to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameter_rule.ex.
Writing SystemParameters to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameters.ex.
Writing Type to clients/service_networking/lib/google_api/service_networking/v1/model/type.ex.
Writing UpdateDnsRecordSetMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/update_dns_record_set_metadata.ex.
Writing UpdateDnsRecordSetRequest to clients/service_networking/lib/google_api/service_networking/v1/model/update_dns_record_set_request.ex.
Writing Usage to clients/service_networking/lib/google_api/service_networking/v1/model/usage.ex.
Writing UsageRule to clients/service_networking/lib/google_api/service_networking/v1/model/usage_rule.ex.
Writing ValidateConsumerConfigRequest to clients/service_networking/lib/google_api/service_networking/v1/model/validate_consumer_config_request.ex.
Writing ValidateConsumerConfigResponse to clients/service_networking/lib/google_api/service_networking/v1/model/validate_consumer_config_response.ex.
Writing Operations to clients/service_networking/lib/google_api/service_networking/v1/api/operations.ex.
Writing Services to clients/service_networking/lib/google_api/service_networking/v1/api/services.ex.
Writing connection.ex.
Writing metadata.ex.
Writing mix.exs
Writing README.md
Writing LICENSE
Writing .gitignore
Writing config/config.exs
Writing test/test_helper.exs

13:33:19.503 [info]  Found only discovery_revision and/or formatting changes. Not significant enough for a PR.
fixing file permissions
2020-06-06 06:33:22,630 synthtool [DEBUG] > Wrote metadata to clients/service_networking/synth.metadata.
DEBUG:synthtool:Wrote metadata to clients/service_networking/synth.metadata.
2020-06-06 06:33:22,659 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 555, in _inner_main
    ).synthesize(base_synth_log_path)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 121, in synthesize
    with open(log_file_path, "rt") as fp:
IsADirectoryError: [Errno 21] Is a directory: '/tmpfs/src/github/synthtool/logs/googleapis/elixir-google-api'

Google internal developers can see the full log here.

Autosynth is still having trouble generating ServiceNetworking. :sob:
Here's the output from running synth.py:
service_networking/v1/model/billing_destination.ex.
Writing CancelOperationRequest to clients/service_networking/lib/google_api/service_networking/v1/model/cancel_operation_request.ex.
Writing Connection to clients/service_networking/lib/google_api/service_networking/v1/model/connection.ex.
Writing ConsumerProject to clients/service_networking/lib/google_api/service_networking/v1/model/consumer_project.ex.
Writing Context to clients/service_networking/lib/google_api/service_networking/v1/model/context.ex.
Writing ContextRule to clients/service_networking/lib/google_api/service_networking/v1/model/context_rule.ex.
Writing Control to clients/service_networking/lib/google_api/service_networking/v1/model/control.ex.
Writing CustomError to clients/service_networking/lib/google_api/service_networking/v1/model/custom_error.ex.
Writing CustomErrorRule to clients/service_networking/lib/google_api/service_networking/v1/model/custom_error_rule.ex.
Writing CustomHttpPattern to clients/service_networking/lib/google_api/service_networking/v1/model/custom_http_pattern.ex.
Writing DisableVpcServiceControlsRequest to clients/service_networking/lib/google_api/service_networking/v1/model/disable_vpc_service_controls_request.ex.
Writing DnsRecordSet to clients/service_networking/lib/google_api/service_networking/v1/model/dns_record_set.ex.
Writing DnsZone to clients/service_networking/lib/google_api/service_networking/v1/model/dns_zone.ex.
Writing Documentation to clients/service_networking/lib/google_api/service_networking/v1/model/documentation.ex.
Writing DocumentationRule to clients/service_networking/lib/google_api/service_networking/v1/model/documentation_rule.ex.
Writing Empty to clients/service_networking/lib/google_api/service_networking/v1/model/empty.ex.
Writing EnableVpcServiceControlsRequest to clients/service_networking/lib/google_api/service_networking/v1/model/enable_vpc_service_controls_request.ex.
Writing Endpoint to clients/service_networking/lib/google_api/service_networking/v1/model/endpoint.ex.
Writing Enum to clients/service_networking/lib/google_api/service_networking/v1/model/enum.ex.
Writing EnumValue to clients/service_networking/lib/google_api/service_networking/v1/model/enum_value.ex.
Writing Field to clients/service_networking/lib/google_api/service_networking/v1/model/field.ex.
Writing GoogleCloudServicenetworkingV1betaSubnetwork to clients/service_networking/lib/google_api/service_networking/v1/model/google_cloud_servicenetworking_v1beta_subnetwork.ex.
Writing Http to clients/service_networking/lib/google_api/service_networking/v1/model/http.ex.
Writing HttpRule to clients/service_networking/lib/google_api/service_networking/v1/model/http_rule.ex.
Writing JwtLocation to clients/service_networking/lib/google_api/service_networking/v1/model/jwt_location.ex.
Writing LabelDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/label_descriptor.ex.
Writing ListConnectionsResponse to clients/service_networking/lib/google_api/service_networking/v1/model/list_connections_response.ex.
Writing ListOperationsResponse to clients/service_networking/lib/google_api/service_networking/v1/model/list_operations_response.ex.
Writing LogDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/log_descriptor.ex.
Writing Logging to clients/service_networking/lib/google_api/service_networking/v1/model/logging.ex.
Writing LoggingDestination to clients/service_networking/lib/google_api/service_networking/v1/model/logging_destination.ex.
Writing Method to clients/service_networking/lib/google_api/service_networking/v1/model/method.ex.
Writing MetricDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/metric_descriptor.ex.
Writing MetricDescriptorMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/metric_descriptor_metadata.ex.
Writing MetricRule to clients/service_networking/lib/google_api/service_networking/v1/model/metric_rule.ex.
Writing Mixin to clients/service_networking/lib/google_api/service_networking/v1/model/mixin.ex.
Writing MonitoredResourceDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/monitored_resource_descriptor.ex.
Writing Monitoring to clients/service_networking/lib/google_api/service_networking/v1/model/monitoring.ex.
Writing MonitoringDestination to clients/service_networking/lib/google_api/service_networking/v1/model/monitoring_destination.ex.
Writing OAuthRequirements to clients/service_networking/lib/google_api/service_networking/v1/model/o_auth_requirements.ex.
Writing Operation to clients/service_networking/lib/google_api/service_networking/v1/model/operation.ex.
Writing Option to clients/service_networking/lib/google_api/service_networking/v1/model/option.ex.
Writing Page to clients/service_networking/lib/google_api/service_networking/v1/model/page.ex.
Writing PolicyBinding to clients/service_networking/lib/google_api/service_networking/v1/model/policy_binding.ex.
Writing Quota to clients/service_networking/lib/google_api/service_networking/v1/model/quota.ex.
Writing QuotaLimit to clients/service_networking/lib/google_api/service_networking/v1/model/quota_limit.ex.
Writing Range to clients/service_networking/lib/google_api/service_networking/v1/model/range.ex.
Writing RangeReservation to clients/service_networking/lib/google_api/service_networking/v1/model/range_reservation.ex.
Writing RemoveDnsRecordSetMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_metadata.ex.
Writing RemoveDnsRecordSetRequest to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_request.ex.
Writing RemoveDnsRecordSetResponse to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_response.ex.
Writing RemoveDnsZoneMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_metadata.ex.
Writing RemoveDnsZoneRequest to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_request.ex.
Writing RemoveDnsZoneResponse to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_response.ex.
Writing Route to clients/service_networking/lib/google_api/service_networking/v1/model/route.ex.
Writing SearchRangeRequest to clients/service_networking/lib/google_api/service_networking/v1/model/search_range_request.ex.
Writing Service to clients/service_networking/lib/google_api/service_networking/v1/model/service.ex.
Writing ServiceIdentity to clients/service_networking/lib/google_api/service_networking/v1/model/service_identity.ex.
Writing SourceContext to clients/service_networking/lib/google_api/service_networking/v1/model/source_context.ex.
Writing SourceInfo to clients/service_networking/lib/google_api/service_networking/v1/model/source_info.ex.
Writing Status to clients/service_networking/lib/google_api/service_networking/v1/model/status.ex.
Writing Subnetwork to clients/service_networking/lib/google_api/service_networking/v1/model/subnetwork.ex.
Writing SystemParameter to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameter.ex.
Writing SystemParameterRule to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameter_rule.ex.
Writing SystemParameters to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameters.ex.
Writing Type to clients/service_networking/lib/google_api/service_networking/v1/model/type.ex.
Writing UpdateDnsRecordSetMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/update_dns_record_set_metadata.ex.
Writing UpdateDnsRecordSetRequest to clients/service_networking/lib/google_api/service_networking/v1/model/update_dns_record_set_request.ex.
Writing Usage to clients/service_networking/lib/google_api/service_networking/v1/model/usage.ex.
Writing UsageRule to clients/service_networking/lib/google_api/service_networking/v1/model/usage_rule.ex.
Writing ValidateConsumerConfigRequest to clients/service_networking/lib/google_api/service_networking/v1/model/validate_consumer_config_request.ex.
Writing ValidateConsumerConfigResponse to clients/service_networking/lib/google_api/service_networking/v1/model/validate_consumer_config_response.ex.
Writing Operations to clients/service_networking/lib/google_api/service_networking/v1/api/operations.ex.
Writing Services to clients/service_networking/lib/google_api/service_networking/v1/api/services.ex.
Writing connection.ex.
Writing metadata.ex.
Writing mix.exs
Writing README.md
Writing LICENSE
Writing .gitignore
Writing config/config.exs
Writing test/test_helper.exs

13:24:06.535 [info]  Found only discovery_revision and/or formatting changes. Not significant enough for a PR.
fixing file permissions
2020-06-07 06:24:09,675 synthtool [DEBUG] > Wrote metadata to clients/service_networking/synth.metadata.
DEBUG:synthtool:Wrote metadata to clients/service_networking/synth.metadata.
2020-06-07 06:24:09,698 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 555, in _inner_main
    ).synthesize(base_synth_log_path)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 121, in synthesize
    with open(log_file_path, "rt") as fp:
IsADirectoryError: [Errno 21] Is a directory: '/tmpfs/src/github/synthtool/logs/googleapis/elixir-google-api'

Google internal developers can see the full log here.

Autosynth is still having trouble generating ServiceNetworking. :sob:
Here's the output from running synth.py:
service_networking/v1/model/billing_destination.ex.
Writing CancelOperationRequest to clients/service_networking/lib/google_api/service_networking/v1/model/cancel_operation_request.ex.
Writing Connection to clients/service_networking/lib/google_api/service_networking/v1/model/connection.ex.
Writing ConsumerProject to clients/service_networking/lib/google_api/service_networking/v1/model/consumer_project.ex.
Writing Context to clients/service_networking/lib/google_api/service_networking/v1/model/context.ex.
Writing ContextRule to clients/service_networking/lib/google_api/service_networking/v1/model/context_rule.ex.
Writing Control to clients/service_networking/lib/google_api/service_networking/v1/model/control.ex.
Writing CustomError to clients/service_networking/lib/google_api/service_networking/v1/model/custom_error.ex.
Writing CustomErrorRule to clients/service_networking/lib/google_api/service_networking/v1/model/custom_error_rule.ex.
Writing CustomHttpPattern to clients/service_networking/lib/google_api/service_networking/v1/model/custom_http_pattern.ex.
Writing DisableVpcServiceControlsRequest to clients/service_networking/lib/google_api/service_networking/v1/model/disable_vpc_service_controls_request.ex.
Writing DnsRecordSet to clients/service_networking/lib/google_api/service_networking/v1/model/dns_record_set.ex.
Writing DnsZone to clients/service_networking/lib/google_api/service_networking/v1/model/dns_zone.ex.
Writing Documentation to clients/service_networking/lib/google_api/service_networking/v1/model/documentation.ex.
Writing DocumentationRule to clients/service_networking/lib/google_api/service_networking/v1/model/documentation_rule.ex.
Writing Empty to clients/service_networking/lib/google_api/service_networking/v1/model/empty.ex.
Writing EnableVpcServiceControlsRequest to clients/service_networking/lib/google_api/service_networking/v1/model/enable_vpc_service_controls_request.ex.
Writing Endpoint to clients/service_networking/lib/google_api/service_networking/v1/model/endpoint.ex.
Writing Enum to clients/service_networking/lib/google_api/service_networking/v1/model/enum.ex.
Writing EnumValue to clients/service_networking/lib/google_api/service_networking/v1/model/enum_value.ex.
Writing Field to clients/service_networking/lib/google_api/service_networking/v1/model/field.ex.
Writing GoogleCloudServicenetworkingV1betaSubnetwork to clients/service_networking/lib/google_api/service_networking/v1/model/google_cloud_servicenetworking_v1beta_subnetwork.ex.
Writing Http to clients/service_networking/lib/google_api/service_networking/v1/model/http.ex.
Writing HttpRule to clients/service_networking/lib/google_api/service_networking/v1/model/http_rule.ex.
Writing JwtLocation to clients/service_networking/lib/google_api/service_networking/v1/model/jwt_location.ex.
Writing LabelDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/label_descriptor.ex.
Writing ListConnectionsResponse to clients/service_networking/lib/google_api/service_networking/v1/model/list_connections_response.ex.
Writing ListOperationsResponse to clients/service_networking/lib/google_api/service_networking/v1/model/list_operations_response.ex.
Writing LogDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/log_descriptor.ex.
Writing Logging to clients/service_networking/lib/google_api/service_networking/v1/model/logging.ex.
Writing LoggingDestination to clients/service_networking/lib/google_api/service_networking/v1/model/logging_destination.ex.
Writing Method to clients/service_networking/lib/google_api/service_networking/v1/model/method.ex.
Writing MetricDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/metric_descriptor.ex.
Writing MetricDescriptorMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/metric_descriptor_metadata.ex.
Writing MetricRule to clients/service_networking/lib/google_api/service_networking/v1/model/metric_rule.ex.
Writing Mixin to clients/service_networking/lib/google_api/service_networking/v1/model/mixin.ex.
Writing MonitoredResourceDescriptor to clients/service_networking/lib/google_api/service_networking/v1/model/monitored_resource_descriptor.ex.
Writing Monitoring to clients/service_networking/lib/google_api/service_networking/v1/model/monitoring.ex.
Writing MonitoringDestination to clients/service_networking/lib/google_api/service_networking/v1/model/monitoring_destination.ex.
Writing OAuthRequirements to clients/service_networking/lib/google_api/service_networking/v1/model/o_auth_requirements.ex.
Writing Operation to clients/service_networking/lib/google_api/service_networking/v1/model/operation.ex.
Writing Option to clients/service_networking/lib/google_api/service_networking/v1/model/option.ex.
Writing Page to clients/service_networking/lib/google_api/service_networking/v1/model/page.ex.
Writing PolicyBinding to clients/service_networking/lib/google_api/service_networking/v1/model/policy_binding.ex.
Writing Quota to clients/service_networking/lib/google_api/service_networking/v1/model/quota.ex.
Writing QuotaLimit to clients/service_networking/lib/google_api/service_networking/v1/model/quota_limit.ex.
Writing Range to clients/service_networking/lib/google_api/service_networking/v1/model/range.ex.
Writing RangeReservation to clients/service_networking/lib/google_api/service_networking/v1/model/range_reservation.ex.
Writing RemoveDnsRecordSetMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_metadata.ex.
Writing RemoveDnsRecordSetRequest to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_request.ex.
Writing RemoveDnsRecordSetResponse to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_record_set_response.ex.
Writing RemoveDnsZoneMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_metadata.ex.
Writing RemoveDnsZoneRequest to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_request.ex.
Writing RemoveDnsZoneResponse to clients/service_networking/lib/google_api/service_networking/v1/model/remove_dns_zone_response.ex.
Writing Route to clients/service_networking/lib/google_api/service_networking/v1/model/route.ex.
Writing SearchRangeRequest to clients/service_networking/lib/google_api/service_networking/v1/model/search_range_request.ex.
Writing Service to clients/service_networking/lib/google_api/service_networking/v1/model/service.ex.
Writing ServiceIdentity to clients/service_networking/lib/google_api/service_networking/v1/model/service_identity.ex.
Writing SourceContext to clients/service_networking/lib/google_api/service_networking/v1/model/source_context.ex.
Writing SourceInfo to clients/service_networking/lib/google_api/service_networking/v1/model/source_info.ex.
Writing Status to clients/service_networking/lib/google_api/service_networking/v1/model/status.ex.
Writing Subnetwork to clients/service_networking/lib/google_api/service_networking/v1/model/subnetwork.ex.
Writing SystemParameter to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameter.ex.
Writing SystemParameterRule to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameter_rule.ex.
Writing SystemParameters to clients/service_networking/lib/google_api/service_networking/v1/model/system_parameters.ex.
Writing Type to clients/service_networking/lib/google_api/service_networking/v1/model/type.ex.
Writing UpdateDnsRecordSetMetadata to clients/service_networking/lib/google_api/service_networking/v1/model/update_dns_record_set_metadata.ex.
Writing UpdateDnsRecordSetRequest to clients/service_networking/lib/google_api/service_networking/v1/model/update_dns_record_set_request.ex.
Writing Usage to clients/service_networking/lib/google_api/service_networking/v1/model/usage.ex.
Writing UsageRule to clients/service_networking/lib/google_api/service_networking/v1/model/usage_rule.ex.
Writing ValidateConsumerConfigRequest to clients/service_networking/lib/google_api/service_networking/v1/model/validate_consumer_config_request.ex.
Writing ValidateConsumerConfigResponse to clients/service_networking/lib/google_api/service_networking/v1/model/validate_consumer_config_response.ex.
Writing Operations to clients/service_networking/lib/google_api/service_networking/v1/api/operations.ex.
Writing Services to clients/service_networking/lib/google_api/service_networking/v1/api/services.ex.
Writing connection.ex.
Writing metadata.ex.
Writing mix.exs
Writing README.md
Writing LICENSE
Writing .gitignore
Writing config/config.exs
Writing test/test_helper.exs

13:24:44.169 [info]  Found only discovery_revision and/or formatting changes. Not significant enough for a PR.
fixing file permissions
2020-06-08 06:24:46,763 synthtool [DEBUG] > Wrote metadata to clients/service_networking/synth.metadata.
DEBUG:synthtool:Wrote metadata to clients/service_networking/synth.metadata.
2020-06-08 06:24:46,793 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 555, in _inner_main
    ).synthesize(base_synth_log_path)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 121, in synthesize
    with open(log_file_path, "rt") as fp:
IsADirectoryError: [Errno 21] Is a directory: '/tmpfs/src/github/synthtool/logs/googleapis/elixir-google-api'

Google internal developers can see the full log here.

Autosynth passed, closing! :green_heart:


Seperate the issue management and wiki as 'plugins'

I'd like to suggest that issue managment and the wiki be pulled out of the core code and added as optional plugins. This will help keep gogs even more trim.
In my use case, I want just a git repository manager with a basic web UI that allows me to create repos, push and view contents of the files.
I've grown tired of gogs development slowing down to a snail pace, but I think this plugin separation would allow gogs to be developed separately and allow it to be more useful to many more developers.

What do you mean by slowing down to a snail pace? I have never had performance issues, even around 2 weeks.

We want to introduce a plugin system, but it's always difficult to drop functionality that is already part of the core.

@geek1011 I actually meant "tired of gogs development slowing down"

You can disable them on the repo settings. @ChangePlaces And we have introduced a unit concept to make a basic of plugin. #947 will let you define a team who only see codes or issues or wiki. Of course, this could be a global options then you could disable all repo issues or wiki.

I know you can disable them per repo, but that's still checking for issues that may be assigned to you. I was envisioning a global removal, rather than a disabling. e.g. the core system is installed, and plugins such as different types of issue trackers could be used depending on your needs, different wiki's etc.

depends on #2222

Is it still feasible? Especially for the issue part, the PR system directly depends on it.

IMHO it's still valid. Somebody would potentially replace the builtin issues with another system similar to jira. And in general gitea should still provide a big plugin system.

But, how to ? If not feasible, is it worth to keep it open?

I agree with @wxiaoguang.
Especially since there are already dedicated issues for plugins like #20126.
The only portion that isn't covered by this issue is the "remove issues and wiki from core" part.
And like @wxiaoguang, I doubt that that is really possible.


basic cli arguments

Currently includes [FILE], --help, --version, --fullscreen & --data-dir.
I've made paths global with a OnceLock. This is more efficient in that it doesn't need to setup and load all the paths multiple times, and was also required for persisting the paths across the whole session.
I would like to do something similar with the config ini which also seems to be loaded at multiple places, so I can expose changing things such as the window sizes. Is that ok with you?
Examples
cacophony -h:
A minimalist and ergonomic MIDI sequencer

Usage: cacophony [OPTIONS] [FILE]

Arguments:
  [FILE]  Open the project from disk

Options:
  -d, --data-dir <DIR>  Directory where Cacophony data files reside [env: CACOPHONY_DATA_DIR=] [default: /home/thomas/r/cacophony/data]
  -f, --fullscreen      Make the window fullscreen [env: CACOPHONY_FULLSCREEN=]
  -h, --help            Print help (see more with '--help')
  -V, --version         Print version

cacophony --help:
A minimalist and ergonomic MIDI sequencer

Usage: cacophony [OPTIONS] [FILE]

Arguments:
  [FILE]
          Open the project from disk

Options:
  -d, --data-dir <DIR>
          Directory where Cacophony data files reside

          Uses './data' if not set

          [env: CACOPHONY_DATA_DIR=]
          [default: /home/thomas/r/cacophony/data]

  -f, --fullscreen
          Make the window fullscreen

          Uses 'fullscreen' under '[RENDER]' in 'config.ini' if not set

          Applied after displaying the splash-screen

          [env: CACOPHONY_FULLSCREEN=]

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

@subalterngames I've got some issues with running the actual program, so I couldn't properly test the file loading in the way I wanted. I'll open some issues for that but I would highly recommend testing my fork yourself before merging.
Closes #4.

@tversteeg

I've got some issues with running the actual program, so I couldn't properly test the file loading in the way I wanted. I'll open some issues for that but I would highly recommend testing my fork yourself before merging.

Thanks for the warning. Can you tell me specifically what doesn't work?

Thanks for the warning. Can you tell me specifically what doesn't work?

Whenever I press any of the F keys I get a super fast flashing red text but it's too fast for me to read. I'll open a issue tomorrow when I get home. It's not related to any of my changes because I also get it on the main branch.

Thanks for the revisions. Could you fix the clippy warnings?

Fixed the clippy warnings.


Added (initial) support for compressing spans

What does this PR do?
Fixes #1847
Checklist

[x] This is an enhancement of existing features, or a new feature in existing plugins

[x] I have updated CHANGELOG.asciidoc
[x] I have added tests that prove my fix is effective or that my feature works
[ ] Added an API method or config option? Document in which version this will be introduced
[ ] I have made corresponding changes to the documentation

:grey_exclamation: Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-02-18T17:18:25.595+0000

Duration: 4 min 12 sec

:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run benchmark tests : Run the benchmark tests.

run jdk compatibility tests : Run the JDK Compatibility tests.

run integration tests : Run the Agent Integration tests.

run end-to-end tests : Run the APM-ITs.

run windows tests : Build & tests on windows.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Thanks for your PR! Note that @jackshirazi is already working on this, so please coordinate before proceeding.
@AlexanderWert has now implemented a status indicator in the corresponding issue (#1847) so that it's easier to spot if something is in progress already.

@jackshirazi You pinged the correct person

/test

@elasticmachine run elasticsearch-ci/docs

Thank You!