[wip] WinPB: Revert Cygwin cache, pin Make to v4.2.1-2

I've taken the Cygwin cache PR ( #1219 ) out, as there were some issues with Git :
Cloning into 'C://openjdk-build'...
fatal: unable to access 'https://github.com/adoptopenjdk/openjdk-build/': SSL: couldn't create a context: error:0E079065:configuration file routines:def_load_bio:missing equal sign

When installed normally, this issue doesn't occur.
I've also pinned Cygwin's Make version to 4.2.1-2 as there was an issue building JDK11 with Make version 4.3<EMAIL_ADDRESS>which causes this issue:
make[3]: *** No rule to make target '/cygdrive/c/workspace/build/src/build/windows-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps', needed by '/cygdrive/c/workspace/build/src/build/windows-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.  Stop.

Put back into work in progress as a VPC run has shown that make isn't getting installed properly.

I should probably mention, the issue I'm having is
<IP_ADDRESS> | FAILED | rc=1 >>
C:/cygwin64/bin/make.exe: error while loading shared libraries: cygguile-2.0-22.dll: cannot open shared object file: No such file or directory

That's after I had to manually extract the folder, so I'll have to redo that task as well

https://ci.adoptopenjdk.net/view/Tooling/job/VagrantPlaybookCheck/450/OS=Win2012,label=infra-vagrant-1/console
Can confirm this works - I know the run didn't succeed, but it was able to fully run the playbook and build JDK11 - I'll just need to tweak testJDKWin.sh. Merging


Some issues with multi-PF support

I found some problems when I wanted to use it to simulate PCIE models with multi-PF.
There is a problem when scanning for devices and functions in pci.py.
First, in line 145, it should be dev.pcie_id = PcieId(self.bus_num, d, f) instead of dev.pcie_id = PcieId(self.bus_num, d, 0).
Second,  in line 152, the dev.header_type has changed in awit dev.setup from val  to val & 0x7f. So the judgment condition of if should be if dev.multifunction == 0:.
Also, there is a mistake in device.py line 93: f.multifunction_devlce = True should be f.multifunction_device = True.
Last, I want to know how to set/change an EP's BAR count/BAR size.
After I fixed the above issue, I set up an UltraScalePlusPcieDevice to have two PFs.
Like this:
        self.dev = UltraScalePlusPcieDevice(
            # configuration options
            pcie_generation=3,
            pcie_link_width=16,
            user_clk_frequency=250e6,
            alignment="dword",
            pf_count=2,
            .......

RC successfully enumerates two PFs, Device tree is:
[00-01]---01.0-[01]-+-00.0/-00.1
Scanned to the BAR of PF0: pci 01:00.0: Mem BAR0 (64-bit) raw: 0xffffffffff00000c, mask: 0x0000000000ffffff, size: 16777216.
But the BAR of PF1 is completely empty, I printed the register read:
Config type 0 read, reg 0x004
print current dev 01:00.1 BAR 0 register is 00
Why PF0 with a right BAR set, and PF1 with no bar set?

Nice work, I'll work on getting that cleaned up.
The BARs have to be configured after creating the IP core instance.  In the Corundum testbenches, it is usually done like so:
    self.dev.functions[0].configure_bar(0, 2**len(dut.core_pcie_inst.axil_ctrl_araddr), ext=True, prefetch=True)
    if hasattr(dut.core_pcie_inst, 'pcie_app_ctrl'):
        self.dev.functions[0].configure_bar(2, 2**len(dut.core_pcie_inst.axil_app_ctrl_araddr), ext=True, prefetch=True)

This will configure one or two BARs on function 0.  If you want to configure BARs on all of the functions, then perhaps that should be wrapped in a loop or similar.

ohh, I forgot this. Thank u

Please try out the git master version of cocotbext-pcie and let me know if that fixes the enumeration issues.

Ok, let’s try.
shroud404@linux:~$ pip show cocotbext-pcie
Name: cocotbext-pcie
Version: 0.2.12
Summary: PCI express simulation framework for cocotb
Home-page: https://github.com/alexforencich/cocotbext-pcie
Author: Alex Forencich
Author-email<EMAIL_ADDRESS>License: MIT
Location: /home/shroud404/.local/lib/python3.9/site-packages
Requires: cocotb, cocotbext-axi
Required-by:

And corundum is the latest.
commit c708bc45cd0d5b44274d238dd9fadd16335f337d (HEAD -> master, origin/master, origin/HEAD)
Author: Alex Forencich<EMAIL_ADDRESS>Date:   Tue Dec 6 17:23:28 2022 -0800

    fpga/mqnic/fb2CG: Update testbench

I will test with VCU118 100G tb.
I changed the test_fpga_core.py, to make it has an UltraScalePlusPcieDevice with two PF.
The pf1 is the same as PF0, with the same msix table set.
Set the configuration of PF1 BAR to be the same as that of PF0.
I add some log info when make_functions like this:
        usp_model.py line 754
        self.make_function()
        self.log.info("EP Device make function[0] done")
        self.log.info("EP Device function[0] first is multifunction_device:%s", self.functions[0].multifunction_device)

Use script to capture the typescript when running the test.
Let's see:
first, set EP
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] Xilinx UltraScale+ PCIe hard IP core configuration:
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00]   PCIe speed: gen 3
     ......
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00]   Enable RC 4 TLP straddling: True
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00]   PF count: 2
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] Current append_function pcie_id is 00:00.0
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] EP Device make function[0] done
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] EP Device function[0] first is multifunction_device:False
     0.00ns INFO     ..raScalePlusPcieFunction[00:00.0] Assigned PCIe ID 00:00.1
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] Current append_function pcie_id is 00:00.1
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] EP evice make function[1] done
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] EP Device function[0] second is multifunction_device:True
     0.00ns INFO     ..cie.UltraScalePlusPcieDevice[00] EP Device function[1] second is multifunction_device:True

next Enumerating
   208.00ns INFO     cocotb.pcie.RootComplex            Enumerating bus 1 device 0
   .......
   219.27ns INFO     ..raScalePlusPcieFunction[00:00.0] Assigned PCIe ID 01:00.0
   219.27ns INFO     ..raScalePlusPcieFunction[00:00.1] Assigned PCIe ID 01:00.1
   230.79ns INFO     cocotb.pcie.RootComplex            Found device at 01:00.0
   ........
   299.17ns INFO     cocotb.pcie.RootComplex            Found function at 01:00.0
   ...
   435.92ns INFO     cocotb.pcie.RootComplex            Enumerate function 01:00.0 BAR0
   ...
   527.09ns INFO     cocotb.pcie.RootComplex            pci 01:00.0: Mem BAR0 (64-bit) raw: 0xffffffffff00000c, mask: 0x0000000000ffffff, size: 16777216
   .......
  1347.60ns INFO     cocotb.pcie.RootComplex            Found function at 01:00.1
  ...
  1484.35ns INFO     cocotb.pcie.RootComplex            Enumerate function 01:00.1 BAR0
  ...
  1575.52ns INFO     cocotb.pcie.RootComplex            pci 01:00.1: Mem BAR0 (64-bit) raw: 0xffffffffff00000c, mask: 0x0000000000ffffff, size: 16777216
  .......
  3161.56ns INFO     cocotb.pcie.RootComplex            Configure function 01:00.0 BAR0
  3207.15ns INFO     cocotb.pcie.RootComplex            Configure function 01:00.1 BAR0
  ......
  3252.73ns INFO     cocotb.pcie.RootComplex            Enumeration complete
  3252.73ns INFO     cocotb.pcie.RootComplex            Device tree:
                                                        [00-01]---01.0-[01]-+-00.0
                                                                            \-00.1

OK, it is work!

Excellent.  I'll cut a release with these fixes relatively soon, let me know if you find any other bugs.


Create a develop branch so every PR isn't a release

This project could really benefit from having a develop branch to merge feature branches into. Currently every PR is treated as a "Release". Instead we could merge several small PR's into develop, like all the dependabot PR's. We could then setup automation to generate what the changelog would look like, create a draft or pre-release in github releases, and of course re-run the test suite with all the combined changes. We could even create the vscode vsix extension file so the changes in develop branch are easy to test. Kinda like our own vscode-groovy-lint insiders program. All of these things except changelog generation are already in place in the the Groovy-Guru extension.
Then when ever you want to make a new release you just merge develop into master/main. This is a stepping stone to completely automated releases because you can just put the type of change as a label on the PR and let the automation figure out the version number. See this Groovy-Guru PR.

I'm used to have released reated by "new release" action on github repo, i'll try to do that someday, exactly like with https://github.com/hardisgroupcom/vscode-sfdx-hardis/blob/master/.github/workflows/deploy-RELEASE.yml

Done, now merge in master do nothing, and creating a release publishes a new version of the vscode extension :)


Add purge_queue function (extension, ext client)

Closes #84. Notice this deletes all messages, not archive them (for equivalence with SQS API). If you want something like archive_all we can do it.


Code cleanup + micro optimisations for Enlight library

1. Why is this change necessary?
 Improving both the quality and performance of several functions can therefore have a great impact.
2. What does this change do, exactly?
Micro optimisations and cleaning up code (most of the time one will help the other)
3. Describe each step to reproduce the issue or behaviour.
4. Please link to the relevant issues (if any).
5. Which documentation changes (if any) need to be made because of this PR?
6. Checklist

[x] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[x] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hey @pascalheidmann,
thank you very much for your contribution! 💙
Yet I'm afraid we have to close this one due to time constraints and limited overall benefit to the product. If this change is still important to you, please create a new issue on https://issues.shopware.com so people can up-vote this feature.
For more information about our contribution guidelines, see https://developers.shopware.com/contributing/contribution-guideline/.


Gradients of the decoder are None

Hey!
When using fcvonv_self_att on a new dataset of mine, all the gradients of the parameters (p.grad) are None for the encoder (for the decoder it's fine).
Any ideas where this might come from?

Glad you figured it out!


fix: Checksum for SQL changes containing dbms attributes can get different values if md5sum column is null

Database object needs to be present in scope when upgradingChecksums because of dbms tags in sql changes. This method is only triggered when md5sum column is null.
Fixes #5618
Impact

[X] Bug fix (non-breaking change which fixes expected existing functionality)
[ ] Enhancement/New feature (adds functionality without impacting existing logic)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

or maybe we can also add an integration test scenario with the same steps that were reported in #5618.

@MalloD12 I think the integration test would be good, could you handle it? In fact we require 4 steps:
Steps to Reproduce:

Run update
Clear the checksums using clearCheckSums
Perform an update
Perform another update

Thanks!

or maybe we can also add an integration test scenario with the same steps that were reported in #5618.

@MalloD12 I think the integration test would be good, could you handle it? In fact we require 4 steps:
Steps to Reproduce:

Run update
Clear the checksums using clearCheckSums
Perform an update
Perform another update

Thanks!

Sure, I'll work on it today.


Add files via upload

Added Zip files for bothe examples

I did this as well as I didn't see your pr, but thanks for the initiative. You rock!


No solar panel was detected?

When i running the app, no solar panel was detected? Is there some problem or display error?

Nobody, nobody, just me?

Looks like there was an error loading the predicted solar array layer - let's see if we can recreate.

@spacefan should be working now - there was a permissions issue with the underlying assets - let us know if the app continues to malfunction.

@spacefan should be working now - there was a permissions issue with the underlying assets - let us know if the app continues to malfunction.

No, the issue still exit... ->_<-

@spacefan should be working now - there was a permissions issue with the underlying assets - let us know if the app continues to malfunction.

Yes, issue resolved now! Great work!


Duplicate projects?

Hello,
I am looking from a library that can transliterate from living indic scripts/languages into the roman script.
I have found 3 python projects that all revolve around indic transliteration:

this repo
libindic/indic-trans
virtualvinodh/aksharamukha

So I was wondering how they compare against one another.
I have seen you had made a pull request at aksharamukha and made that comment (#74) back in 2022:

There is a wrapper on aksharamukha at indic_transliteration/__init__.py which I found useful in many scripts (aksharamukha does a much superior job with certain scripts - like tamiL) - so placed it here. Ideally it should go to aksharamukha (if @virtualvinodh agrees to have a contrib module or the like, I'll move it there).

So I guess you must be familiar with the other projects as well.
Is there some specific scripts/languages where aksharamukha or indic-trans still outperform this library? Or some useful feature that one lib has but the others don't?
@irshadbhat

aksharamukha is more thorough (handles more corner cases correctly).
Not familiar with indic-trans.

Thanks, good to know.
So–out of curiosity–why do you keep working on this repo instead of forking aksharamukha and improving it further?

So–out of curiosity–why do you keep working on this repo instead of forking aksharamukha and improving it further?

I'm more familiar with this code; it works fine for most indian languages, plus have control.


enabling UART2 breaks esp32-proxy

Reproduction steps
https://github.com/mangdangroboticsclub/mini_pupper_2_bsp/pull/43 breaks esp32-proxy
Actual results
servos do not move
Expected results
servos will move
Version
Mini Pupper 2
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

I tested the bsp features based on readme and have not found any issues yet.
Let's double-confirm later.

actually it turns out that this is related to the order how services are started and will be fixed. Please close this issue


Azure AI Translation Document Readme issues

Section link:

Suggestion:
1.Add link for [azure_identity]: https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/identity/Azure.Identity/README.md
2.Add link for [custom_subdomain]: https://docs.microsoft.com/en-us/azure/cognitive-services/authentication?tabs=powershell#create-a-resource-with-a-custom-subdomain
3.Update link to https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/identity/Azure.Identity/README.md#defaultazurecredential
4. Add link for [register_aad_app]: https://docs.microsoft.com/en-us/azure/app-service/configure-authentication-provider-aad#-configure-with-advanced-settings
5. Add link for [aad_grant_access]: https://docs.microsoft.com/en-us/powershell/module/az.Resources/New-azRoleAssignment?view=azps-1.8.0
@jongio for notification.

ohhh this is awesome! thanks @Tianyun-G for finding this!


Small IsFinalRequest cleanup

Instead of adding a FinalAttr directly let's have IsFinalRequest determine that lazy property and property wrapper storage is final.
This fixes a bug, apparently only on master and not swift-5.1-branch, where the property wrapper backing storage was not final.

@swift-ci Please smoke test

@swift-ci Please test source compatibility

@swift-ci Please smoke test

@swift-ci Please test source compatibility

https://github.com/apple/swift-lldb/pull/1676
@swift-ci Please smoke test macOS

@swift-ci Please smoke test Linux


NaudioSourceReader.SeekToSecondInCaseIfRequired not working properly on stereo channels

I spotted issue while I was performing recognition of played songs on internet radio stream recording.
In case when fingerprint recognition was done on original recording (128kbps / 44,1kBps / 2ch) detected song duration was approximately two times longer as actual song in recorded file. Also only half of recording file was analyzed.
When performing recognition tests on recorded file which was transcoded in mono channel recognition tests were OK.
I solved the problem with modification in method SeekToSecondInCaseIfRequired in NaudioSourceReader as shown below:
private void SeekToSecondInCaseIfRequired(int startAtSecond, WaveStream stream)
{
  if (startAtSecond > 0)
  {
    // Original code
    // int actualSampleRate = stream.WaveFormat.SampleRate;
    // int bitsPerSample = stream.WaveFormat.BitsPerSample;
    // stream.Seek(actualSampleRate * bitsPerSample / 8 * startAtSecond, System.IO.SeekOrigin.Begin);

    // Code that fixed the problem
    stream.CurrentTime = stream.CurrentTime.Add(new System.TimeSpan(0, 0, 0, startAtSecond, 0));
  }
}

Now I’m getting equal results regardless how many audio channels are present in file being analyzed.

Not relevant anymore.


Problem with site

Is the official site down? I cannot access the documentary of this module.

The site is down!! Is there any place I can get the documentation?

I need it ASAP too

Using Wayback Machine, the latest crawl is from December 3, 2018. Depending what you’re needing it might have it there.
https://web.archive.org/web/20181203064336/http://python.dronekit.io/

Why is the site down? Is that project desserted?

No it is not deserted, it is going through some hosting modification of sorts.
Lots of people on this.

Do you know how long it will take to restore the site?

@proficnc , sorry to bother you, do you know any more about the hosting mods, it would be good to know who was doing them and if they could provide an update to us, it would help a lot,
thanks


Missing codicons.css file

I'm trying to incorporate codicons into a webview for a VSCode extension. The webview sample shows to import a codicons.cs file in the html head, however this file does not exist.
I'm using @vscode/codicons 0.0.35.

Hi! Where did you find these instructions?


[DF] Low Priority Fixes

#211

If the ambassador is also a manager, it doesn't see the ambassador menu options such as community requests. Test account Ambassador 1 has that issue.
Beneficiary sees both countdown and no funds warning. The no funds warning should be at the top of the page if the countdown is active. See design.
Test account Council member 2, that is also ambassador and manager, doesn't see council member pages on sidebar, neither ambassador, only manager

@obernardovieira

the sidebar bugs are fixed on the high priority fixes PR that isn't yet merged.
i'll fix the countdown + no funds, didn't check design, sorry 😅

I did a claim with "beneficiary 4" https://alfajores-blockscout.celo-testnet.org/tx/0xea16720c247f1a73f9f5e8d352d7ac04847d245e75acdca76d5bbb4ce9ce84ff/token-transfers. As beneficiary I see the community has no funds, but i don't see any countdown.

@obernardovieira
from the devs call we've made I understood that if the community has no funds it doesn't know when it will have funds again. it can be days, weeks, etc... when that happens, the user has no countdown.
basically what I understood was:
hasFunds and isClaimable: show claim btn
hasFunds and isClaimable: false: show countdown
hasFunds: false: doesn't show countdown (whether isClaimable is true or not) because even if isClaimable is true there are no funds, so, the beneficiary cant claim even when the countdown ends
from what you've commented, you want to show the countdown when hasfunds: false and isClaimable: true (the community has no funds)?
sorry for the long message and for my misconception once again 😣


DFT output for positive mirror not correct length

The full DFT output should be of length equal to the signal length. The mirrored (only positive) length should be equal to (length of signal)/2 + 1. But the mirrored output length is computed as (length of signal)/2.

Fixed in commit 3c6be691c15549fa9ee4a57d4ebd082170489f88.


There is a problem with initial-page props and gotoPage method

Hi, I want to bring back the state from my storage after refresh the page.
So I actually make the vueTable loadOnStart prop to false and then after I get all my storage information I just want to loadData().
Ive noticed the bug.
When Vuetable is initialized its initalPage state is taken from props and passed to data directly with:
currentPage: this.initialPage,

So after we want to change this prop or just change initial page we would need to destroy this instance and re-render it again to get the state from props. Especially when we dynamically changed this props with Vuex or something.
GoToPage method cannot be actually used for init this current page because without loading the state we have no last_page property on tablePagination. "TypeError: Cannot read property 'last_page' of null"
gotoPage (page) {
  if (page != this.currentPage && (page > 0 && page <= this.tablePagination.last_page)) {
    this.currentPage = page
    this.loadData()
  }

First init state && watcher for the initialPage would be good solution. Or method where we can pass this currentPage property. Then we would need to check if there is page given in our response from server. And if its not just change currentPage to last_page or something.
Doing second call just to change the page is not best solution I guess... especially if we are making call on bigger DB.

@lrembacz I guess you're using Data mode (:api-mode="false") together with VuetablePagination component, am I right?

@ratiw No I am actually using :api-mode="true".
Check my JSFiddle example with same action. Passing props as data, its not reactive.
https://jsfiddle.net/4x5t7Lnw/
You would need watcher actually.
https://jsfiddle.net/4x5t7Lnw/2/

@ratiw Yes, watcher was just example but it's not good enough in this solution.
The problem about initialPage is even much deeper. This problem exists for every data/prop used once.
Unfortunately I don't know the pattern for this solution and can't see any on documentation page.
Initial page prop is actually used once so I  think we don't need to change it reactively.
For now I just directly set this currentPage before loadData() with
this.$refs.vuetable.currentPage = this.initialPage;

But its not good as well.
Maybe just additional data variable, like inited  would be the option here as well. So this data state would determinate if we were taking currentPage or initialPage for first call.
For now I think method to change the sate programmatically is good solution I guess. Or just the set of setters to change some initial data before load from DB.


Pass repo name to dnf-json and use it in dnf-json

dnf-json: do not use reponame as repoid
rpmmd: pass repo name to dnf-json
dnf-json: use repository name from the request if provided
rpmmd: rename toDNFRepoConfig() argument i -> repoID
osbuild-pipeline: use repo name from the request if provided

This looks great to me!
Just to double check that I understood the overall intention: we now include the proper repo name for UI/logging purposes. However, internally we still use repoid, so the name is just an opaque value that does not need to be unique or on any particular form etc?

Correct.
The repoid is still always a unique number and it is what gets passed to dnf-json for any repo. The reply from dnf-json contains a list of packages with information from which repo they come from, specified by repoid. In the past we were not passing reponame to dnf-json, only the repoid. If the reponame is not set, it defaults to repoid in DNF.


Update current user data from DB on start of every session

Link with the get user details API.

#69 , Nice


bug: longhorn manager log is flooded by Enqueuing share manager for pod when using RWX

Even when there is no activity happening, the log is flood by the following (from every Longhorn manager) while using the RWX:
[longhorn-manager-qvnmb] time="2020-12-12T07:04:11Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-3 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] <IP_ADDRESS> - - [12/Dec/2020:07:04:22 +0000] "GET /v1/volumes/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 HTTP/1.1" 200 6831 "" "Go-http-client/1.1"
[longhorn-manager-2dfth] <IP_ADDRESS> - - [12/Dec/2020:07:04:30 +0000] "POST /v1/volumes/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724?action=snapshotList HTTP/1.1" 200 910 "http://longhorn-ui.longhorn-system.<IP_ADDRESS>.xip.io/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
[longhorn-manager-qvnmb] time="2020-12-12T07:04:33Z" level=debug msg="Enqueuing share manager for pod" controller=longhorn-share-manager node=yasker-longhorn-dev-3 pod=share-manager-pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 shareManager=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-jwtbb] time="2020-12-12T07:04:39Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-1 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-jwtbb] time="2020-12-12T07:04:39Z" level=debug msg="Enqueuing share manager for pod" controller=longhorn-share-manager node=yasker-longhorn-dev-1 pod=share-manager-pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 shareManager=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] time="2020-12-12T07:04:40Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-2 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] time="2020-12-12T07:04:40Z" level=debug msg="Enqueuing share manager for pod" controller=longhorn-share-manager node=yasker-longhorn-dev-2 pod=share-manager-pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 shareManager=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-qvnmb] time="2020-12-12T07:04:41Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-3 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] <IP_ADDRESS> - - [12/Dec/2020:07:04:50 +0000] "POST /v1/volumes/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724?action=snapshotList HTTP/1.1" 200 910 "http://longhorn-ui.longhorn-system.<IP_ADDRESS>.xip.io/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
[longhorn-manager-qvnmb] time="2020-12-12T07:05:03Z" level=debug msg="Enqueuing share manager for pod" controller=longhorn-share-manager node=yasker-longhorn-dev-3 pod=share-manager-pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 shareManager=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-jwtbb] time="2020-12-12T07:05:09Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-1 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-jwtbb] time="2020-12-12T07:05:09Z" level=debug msg="Enqueuing share manager for pod" controller=longhorn-share-manager node=yasker-longhorn-dev-1 pod=share-manager-pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 shareManager=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] time="2020-12-12T07:05:10Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-2 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] <IP_ADDRESS> - - [12/Dec/2020:07:05:10 +0000] "POST /v1/volumes/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724?action=snapshotList HTTP/1.1" 200 910 "http://longhorn-ui.longhorn-system.<IP_ADDRESS>.xip.io/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
[longhorn-manager-2dfth] time="2020-12-12T07:05:10Z" level=debug msg="Enqueuing share manager for pod" controller=longhorn-share-manager node=yasker-longhorn-dev-2 pod=share-manager-pvc-63f4015d-b441-48c7-b7cb-bba74af0d724 shareManager=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-qvnmb] time="2020-12-12T07:05:11Z" level=debug msg="Enqueuing share manager for volume" accessMode=rwx controller=longhorn-share-manager frontend=blockdev node=yasker-longhorn-dev-3 owner=yasker-longhorn-dev-2 shareEndpoint="nfs://<IP_ADDRESS>/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724" shareState=running state=attached volume=pvc-63f4015d-b441-48c7-b7cb-bba74af0d724
[longhorn-manager-2dfth] <IP_ADDRESS> - - [12/Dec/2020:07:05:30 +0000] "POST /v1/volumes/pvc-63f4015d-b441-48c7-b7cb-bba74af0d724?action=snapshotList HTTP/1.1" 200 910 "http://longhorn-ui.longhorn-system.<IP_ADDRESS>.xip.io/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

Verified on master-12/14/2020. No more log spam.


Feature: Support show create trigger SQL statement.

What would you like to be added:
Why is this needed:
https://dev.mysql.com/doc/refman/8.0/en/show-create-trigger.html
SHOW CREATE TRIGGER trigger_name

Please assign this to me.  Before doing this feature,  issue #237 should be done.


Add oVirt platform status

Add oVirt platform status
enhancement: https://github.com/openshift/enhancements/pull/61

Please link the enhancement to this PR so we can understand the impact this has on 4.3

Please link the enhancement to this PR so we can understand the impact this has on 4.3

Can you link the enhancement please

/approve
Seems inline with other platform definitions.
@abhinavdahiya PTAL

/lgtm


Add dd apm tracing for server and db

Description
Actually, we do need to instrument code to trace http handler and DB
Motivation and Context

How Has This Been Tested?

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Have you considered using opencensus to make this vendor agnostic?

@tmc good point! I will try, which opentracing/opencensus compatible package do you recommend?


Make DfApp

Make main class of framework - DfApp.
This class need to provide methods for work with:

Routing
Logging
Configuration
Components includes(DfMysql..)

Make auto load and start develop class(https://github.com/daitel/framework/commit/eecb51c9aaaa68c9e3c18501b4ee3cbdc92c5085)


fix: set token cookie secure and samesite per config

Enhance Cookie Security Settings
Description
This pull request enhances the security of session cookies by adding samesite and secure attributes to cookie settings within the authentication and session management functions. These changes aim to improve the application's resilience against cross-site request forgery (CSRF) attacks and ensure that cookies are only transmitted over secure channels.
Changelog
Added

Included samesite and secure attributes to the cookie settings in the following functions:

get_session_user
signin
signup
oauth_callback

Changed

Updated the auths.py and main.py files to include the new cookie attributes.

Security

Enhanced session cookies with samesite and secure attributes to mitigate CSRF attacks and ensure cookies are transmitted securely.

Note to first-time contributors
Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Pull Request Checklist

[x] Target branch: This pull request targets the dev branch.
[x] Description: Provided a concise description of the changes made in this pull request.
[x] Changelog: Ensured a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[ ] Documentation: Updated relevant documentation Open WebUI Docs, or other documentation sources.
[x] Dependencies: Checked for any new dependencies and updated the dependency versions in the documentation.
[x] Testing: Ran sufficient manual tests for validating the changes.
[x] Code review: Performed a self-review of the code, addressing any coding standard issues and ensuring adherence to the project's coding standards.
[x] Prefix: Prefixed the pull request title using one of the following:

BREAKING CHANGE: Significant changes that may affect compatibility
build: Changes that affect the build system or external dependencies
ci: Changes to our continuous integration processes or workflows
chore: Refactor, cleanup, or other non-functional code changes
docs: Documentation update or addition
feat: Introduces a new feature or enhancement to the codebase
fix: Bug fix or error correction
i18n: Internationalization or localization changes
perf: Performance improvement
refactor: Code restructuring for better maintainability, readability, or scalability
style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
test: Adding missing tests or correcting existing tests
WIP: Work in progress, a temporary label for incomplete or ongoing work

Additional Information

Reference issues: #4233

Screenshots or Videos

NA

This pull request aims to improve the security of session management within the application. If there are any questions or further clarifications needed, please let me know. Thank you for reviewing and considering these changes.

Screenshot

LGTM, Thanks!


Consistenty use spaces for indentation

Also strip trailing whitespace and ensure a single final new line.
No functional changes intended.

Looking good.

I'm going to do @ejelly's job of merging this :)

@dblock Oh it is? 8)


Issue in Signed Event Webhook - PHP

Issue Summary
I have followed the steps given in https://sendgrid.com/docs/for-developers/tracking-events/getting-started-event-webhook-security-features/ for PHP and used library mentioned in https://github.com/sendgrid/sendgrid-php/tree/main/lib/eventwebhook similarly checked unittest https://github.com/sendgrid/sendgrid-php/blob/main/test/unit/EventWebhookTest.php for event signature verification
Steps to Reproduce

Went to https://app.sendgrid.com/settings/mail_settings and did test your integration with the ngrok url.
Got request with header on ngrok url
Output from verifySignature is false.

Code Snippet
$timestamp = $req->header("X-Twilio-Email-Event-Webhook-Timestamp");
$signature = $req->header("X-Twilio-Email-Event-Webhook-Signature");
$verifycationKey = "<Verification Key>";
$body = \json_encode($req->all())."\r\n";
\Log::info("+++++++++++++++++++");
\Log::info($body);
\Log::info($timestamp);
\Log::info($signature);

$event = new EventWebhook;

$publicKey = $event->convertPublicKeyToECDSA($verifycationKey);
\Log::info(serialize($publicKey));
$flag =$event->verifySignature($publicKey, $body, $signature, $timestamp);
\Log::info("flag====> ".$flag);
\Log::info("+++++++++++++++++++");

Technical details:

sendgrid-php version: 7.9
php version: 7.2

Yes we are having multi-event webhook. As per your comment we have modified the code as follows still not luck.
`
$timestamp = $req->header("X-Twilio-Email-Event-Webhook-Timestamp");
$signature = $req->header("X-Twilio-Email-Event-Webhook-Signature");
$verifycationKey = "";
$body = \json_encode($req->all());
$body = str_replace("},{", "},\r\n{",$body)."\r\n";
\Log::info("+++++++++++++++++++");
\Log::info($body);
\Log::info($timestamp);
\Log::info($signature);
$event = new EventWebhook;
$publicKey = $event->convertPublicKeyToECDSA($verifycationKey);
\Log::info(serialize($publicKey));
$flag =$event->verifySignature($publicKey, $body, $signature, $timestamp);
\Log::info("flag====> ".$flag);
\Log::info("+++++++++++++++++++");
`

@iprashant have you tried with the raw body?
$body = $request->getContent()
$body = str_replace("},{", "},\r\n{",$body)."\r\n";

Hi Swetha,
We used PHP file_get_contents('php://input'); still no luck.
Thanks,

Hi Shwetha,
I have used PHP file_get_contents('php://input'); still no luck.
Thanks,

OK, we have an internal ticket to investigate the multi-event webhook scenario. Marking this as a bug in the meantime.
This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.

This replacement worked for me on multi-event payloads instead of the one above:
$bodyContent = str_replace(
    "},\n{", // The raw string comes in with newlines in this spot
    "},\r\n{",
    $body // $request->getContent() can be used here for Laravel apps
)."\r\n";

Thank you for taking the time to share your solution @peter-at-bpt!

Hey there, we at fonQ are running into the same issue with the signature verification and the solution of @peter-at-bpt unfortunately does not seem to work. What is the status of your internal investigation?

webhook signed webhook getting false for function = isValidSignature
$publicKey = 'my key';
$input = '[{"DRUPAL_ROOT":"/home/gems2/git/harmony","OURHTTP_HOST":"gemslocal2.artofliving.org","OURREQUEST_URI":"/ca/civicrm/event/register","contact_id":"24495","countryCode":"ca","email":"paul@yopmail.com","event":"processed","newHash":"87858ab1e574e0be","newJobId":"39003277","newQueueId":11312,"newemail_id":"13842","send_at":0,"sg_event_id":"cHJvY2Vzc2VkLTIyOTY4OTQyLWlhcXlDLWhlUW5TRnc4LWtsX0dTOWctMA","sg_message_id":"iaqyC-heQnSFw8-kl_GS9g.filterdrecv-7489454b79-khx4b-1-611E028F-2B.0","smtp-id":"iaqyC-heQnSFw8-kl_GS9g@geopod-ismtpd-6-0","timestamp":1629356687,"uniqueParam":"b.39003277.11312.87858ab1e574e0be@newsletter.india.artofliving.org"},{"DRUPAL_ROOT":"/home/gems2/git/harmony","OURHTTP_HOST":"gemslocal2.artofliving.org","OURREQUEST_URI":"/ca/civicrm/event/register","contact_id":"24495","countryCode":"ca","email":"paul@yopmail.com","event":"delivered","ip":"<IP_ADDRESS>","newHash":"87858ab1e574e0be","newJobId":"39003277","newQueueId":11312,"newemail_id":"13842","response":"250 mail saved","sg_event_id":"ZGVsaXZlcmVkLTAtMjI5Njg5NDItaWFxeUMtaGVRblNGdzgta2xfR1M5Zy0w","sg_message_id":"iaqyC-heQnSFw8-kl_GS9g.filterdrecv-7489454b79-khx4b-1-611E028F-2B.0","smtp-id":"iaqyC-heQnSFw8-kl_GS9g<EMAIL_ADDRESS>$headerStringValue = $_SERVER['HTTP_X_TWILIO_EMAIL_EVENT_WEBHOOK_SIGNATURE'];
$timestamp = $_SERVER['HTTP_X_TWILIO_EMAIL_EVENT_WEBHOOK_TIMESTAMP'];
and i am calling function like below
$result =   isValidSignature($publicKey,$input,$headerStringValue,$timestamp);
i am getting result as false- but all values i am passing fine and also php library also fine.
help me to proceed , whats wrong in above code or any other setting i need to enable?
i have tested in postman by passing these details.
when i debugged - i am getting false from this function :
$success = openssl_verify($message, $signature->toDer(), $publicKey->openSslPublicKey, OPENSSL_ALGO_SHA256);
here its false , please help


cargo-public-api: 0.32.0 -> 0.33.0

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Priorities
Add a :+1: reaction to pull requests you find important.

Not sure whether we want to backport such things... on one hand there's a bugfix in there, on the other it is not critical (from what I understand): https://github.com/Enselic/cargo-public-api/releases/tag/v0.33.0
Also, most users might import this from unstable anyways (or am I standing alone here?) :laughing:

Result of nixpkgs-review pr 270860 run on x86_64-linux 1

1 package built:

cargo-public-api

Result of nixpkgs-review pr 270860 run on x86_64-linux 1

1 package built:

cargo-public-api


[Feature] Driver License / DMV Drivers Test (Police CMDS aswell)

Buy Driver License / DMV Drivers Test
Police:
Check for driver license
Check for license plate on vehicles to see if they registered or not registered(NPC Vehicles / Stolen), and who they belong to.

It's already possible.

So it is already possible to check driver license and license plate?

No idea what is the DMV Drivers test, but you can just create a driver license item (police can check items) and plates are already bound to the user, it's the registration number. Again, it's not an help section, you should use the forum topic or the discord.


Scipy minimize fail to iterate

Hi,
I am litteraly going crazy with Scipy.minimize.
I am simply trying to fit a certain list to a set of parameters representing a bezier curve.
My input is the "curveToFitArray". I have an initial guess "x0" that I cut in half to have bezier points coordinates in "Y" and "Z". I want to find the parameters that minimize the error between the curve generated by my bezier parametrization and the real curve this way:
#!/home/charles/miniconda3/envs/default/bin/python
from scipy.optimize import minimize
from scipy.interpolate import interp1d
import numpy as np
from scipy.special import comb
def bernstein_poly(i, n, t):
	return comb(n, i) * ( t**(n-i) ) * (1 - t)**i

def bezier_curve(points, nTimes=50):
	nPoints = len(points)
	xPoints = np.array([p[0] for p in points])
	yPoints = np.array([p[1] for p in points])

	t = 0.5 * (1 - np.cos(np.linspace(0,np.pi,nTimes)))
	polynomial_array = np.array([bernstein_poly(i, nPoints-1, t) for i in range(0, nPoints)])

	xvals = np.dot(xPoints, polynomial_array)
	yvals = np.dot(yPoints, polynomial_array)

	return [list(c) for c in zip(xvals,yvals)]
def bspline(p,scale=[1.0,1.0],U='Y',V='X',nPoints=50):
	point_list = [[u*scale[0],v*scale[1]] for u,v in zip(p[U],p[V])]
	BSpline = bezier_curve(point_list,nPoints)
	#pts_BSpline = np.array(BSpline).T
	return sorted(BSpline, key=lambda x: x[0])

def Distance(curve):
    distance = np.cumsum(np.sqrt(np.sum(np.diff(curve, axis=0)**2, axis=1)))
    distance = np.insert(distance, 0, 0)/distance[-1]
    return distance
def divide(curve, n):
    distance = Distance(curve)
    alpha = np.linspace(0, 1, n)
    interpolation = interp1d(distance, curve, kind='linear', axis=0)
    interp_sort = sorted([[s, *c] for s, c in zip(alpha, interpolation(alpha))], key=lambda x: x[0])
    return [c[1:] for c in interp_sort]

def parseParameters(x):
    d = {}
    d["Y"] = list(x[0:4])
    d["Z"] = list(x[4:])
    return d

def objective(x,curveToFit):

    parameters = parseParameters(x)

    curveEval = divide(bspline(p = parameters,U='Y',V='Z',nPoints=len(curveToFit)),len(curveToFit))
    curveEval = np.array(curveEval)
    curveToFit = np.array(curveToFit)

    print("curveToFit",curveToFit)
    print("curveEval",curveEval)
    error = np.sqrt((np.subtract(curveEval, curveToFit)** 2))
    print("error",error)
    error = np.sum(error)

    return error

curveToFit = [[0.0, 0.0], [0.08462454350227121, -0.0006072162233802024], [0.16920310091996732, -0.0034025456809973417], [0.2533683125896774, -0.012003143737867554], [0.337444, -0.021644]]

x0 = [0.0, 0.050616600000000005, 0.253083, 0.337444, 0.0, 0.0, -0.021644, -0.021644]

bounds = [[0.0, 0.0], [-0.015, 0.35244400000000004], [-0.015, 0.35244400000000004], [0.337444, 0.337444],[0.0, 0.0], [-0.036643999999999996, 0.015], [-0.036643999999999996, 0.015], [-0.021644, -0.021644]]

result = minimize(
    fun = objective,
    x0 = x0,
    bounds = bounds,
    method='L-BFGS-B', # L-BFGS-B, SLSQP
    args=(curveToFit),
    options={'disp':True}
)

print(result)

I had it working nicely a few months ago. Then did problably some modifications on my code without checking the fitting was still working. Now I cant remember what I changed to make it stop working...
The algorithm just doesnt iterate and return a result very close to x0 everytime
Thanks a lot

Without looking too closely at the particular problem, here are some thoughts:

This doesn't smell like a bug yet - an algorithm not iterating on a particular problem is not necessarily a bug, as x0 could near a stationary point - so stackoverflow would be a better place to start. We use GitHub for bugs and feature requests.
Try different methods
Try a different guess, or, if there is a reason to suspect this is a good guess, add noise to the guess
Try a global optimizer, e.g. differential_evolution. Shouldn't be too hard to adapt your code to that.

I'll leave this open for now. If it doesn't get too much attention and there is no more evidence that it's a bug, I'll close. Sound good?

Hi mdhaber,
I didnt say it was necessary a bug. The thing is I have been used the algorithm (with L-BFGS) forever and it suddenly stopped to work and I can't explain why. I have spent days on trying to solve the problem. It's certainly related to something I have changed or that has changed in scipy.
I just can't afford in terms of computational time to use a global optimizer.
I am just posting here because I run out of solutions to make this work again and I really need help on this
thank you

I'm guessing that the reason it doesn't iterate is because there are four parameters whose lower bounds are equal to the upper bounds. Can you factor those out? The code that calculates the gradients strictly obeys the bounds, and it's difficult to calculate a finite difference if the lower and upper bounds are equal.

Hi andrew,
This is not the reason because I am using this method in many other applications and this always works. I can't do with another option because I need to have a fixed number of parameters.

What is the output of your example? Are any Exceptions raised?

>>> from scipy.optimize import minimize, rosen
>>> # last parameter bounds are equal
>>> bounds = [(0, 10), (0, 10), (2, 2)]

>>> minimize(rosen, (2, 2, 2), method='L-BFGS-B', bounds=bounds)
/Users/andrew/miniconda3/envs/dev3/lib/python3.8/site-packages/scipy/optimize/_numdiff.py:519: RuntimeWarning: invalid value encountered in true_divide
  J_transposed[i] = df / dx
      fun: 402.0
 hess_inv: <3x3 LbfgsInvHessProduct with dtype=float64>
      jac: array([1602.00001865, 1202.00001723,           nan])
  message: b'ABNORMAL_TERMINATION_IN_LNSRCH'
     nfev: 8
      nit: 0
     njev: 2
   status: 2
  success: False
        x: array([2., 2., 2.])

@carlitador do you happen to know the last version of SciPy in which l-bfgs-b was working for you?

>>> from scipy.optimize import minimize, rosen
>>> # last parameter bounds are equal
>>> bounds = [(0, 10), (0, 10), (2, 2)]

>>> minimize(rosen, (2, 2, 2), method='L-BFGS-B', bounds=bounds)
/Users/andrew/miniconda3/envs/dev3/lib/python3.8/site-packages/scipy/optimize/_numdiff.py:519: RuntimeWarning: invalid value encountered in true_divide
  J_transposed[i] = df / dx
      fun: 402.0
 hess_inv: <3x3 LbfgsInvHessProduct with dtype=float64>
      jac: array([1602.00001865, 1202.00001723,           nan])
  message: b'ABNORMAL_TERMINATION_IN_LNSRCH'
     nfev: 8
      nit: 0
     njev: 2
   status: 2
  success: False
        x: array([2., 2., 2.])

Please understand this is not the issue. I am using this equal bounds for many other projects and its been working well forever.
Here is the output of my problem
/home/charles/miniconda3/envs/default/lib/python3.8/site-packages/scipy/optimize/_numdiff.py:519: RuntimeWarning: invalid value encountered in true_divide
  J_transposed[i] = df / dx
RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =            8     M =           10

At X0         4 variables are exactly at the bounds

At iterate    0    f=  1.97255D-02    |proj g|=          NaN

At iterate    1    f=  1.97255D-02    |proj g|=          NaN

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
    8      1      3      3     0     6         NaN   1.973D-02
  F =   1.9725495590310269E-002

CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH
      fun: 0.01972549559031027
 hess_inv: <8x8 LbfgsInvHessProduct with dtype=float64>
      jac: array([        nan, -0.0637971 , -0.06698817,         nan,         nan,
       -0.87670644, -0.93680716,         nan])
  message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
     nfev: 27
      nit: 1
     njev: 3
   status: 0
  success: True
        x: array([ 0.       ,  0.0506166,  0.253083 ,  0.337444 ,  0.       ,
        0.       , -0.021644 , -0.021644 ])

So while this may works, putting the values at the bounds is not what you want and it may gives some NaNs. What about you don't give the values to the function (but e.g. use partial to give them as constants)? While it does not "solve" the break in the code, I think setting the bounds like this is also happen to break for different reasons.

I experienced a very similar problem, at least I think so. So you probably know the book
IPython Interactive Computing and Visualization Cookbook, Second Edition (2018), by Cyrille Rossant
Recently when testing the examples given there and in particular
https://ipython-books.github.io/94-finding-the-equilibrium-state-of-a-physical-system-by-minimizing-its-potential-energy/
I found that example doesn't work.
L-BFGS fails to fined minima of the problem with bounds, and just says that inital configuration is a solution to the problem.
When changing the method to Powell's everything works.
I message Cyrille and he told me that previously it was working, but now indeed only Powell's works (the program itself remains unchanged)
https://github.com/ipython-books/cookbook-2nd-code/pull/4
So it is quite possible that behavior of the method implementation has changed itself.

i am getting this same issue with L-BFGS-B not progressing (NaN gradient) if any bounds have equal upper/lower, however I am sure the same approach was working in previous scipy versions with L-BFGS-B. The docs generally suggests using bounds exactly this way to keep parameters fixed, so it would be great if it could be fixed again. thanks :)

I believe this issue should be fixed with #13096, with the fix being available in scipy 1.8.
If I understand correctly the issue arises because of problems estimating the gradient when a lower bound was equal to an upper bound. Changes made to strictly obey bounds during finite difference differentiation caused approx_derivative to start returning grads containing NaN. L-BFGS-B (and a couple of others) didn't like that. The fix in #13096 factorises parameters with equal bounds out.


update docs for Submission.link_flair_template_id

Closes #1700
Feature Summary and Justification
Docs reflect that link_flair_template_id attribute does not return None when a link flair isn't present

Just a couple of things:

Could you add Closes #1700 at the beginning of the description/comment so it'll automatically close that when this gets merged?
Could you add yourself to the documentation section in the AUTHORS.rst file?

Does that work @LilSpazJoekp ?

It looks like the checks are failing now.

Whoops lol. Hope that works. I can't be bothered to clone this right now

That works, thank you!🎉


Add open folder button

closes #48

Yeah, looks good.
I just pushed the button to the end of the row for style ;)

Yeah, looks good.
I just pushed the button to the end of the row for style ;)


Changes the headVersion of the HandshakeRequest message to be a string

Summary
Changes the headVersion to be a string

This would allow passing in whatever Docker-tag as a version from the SDKs, but would have big implications on the EventStore because it bubbles down into the ExecutionContext. Therefore we've decided to leave this idea for now.


[post] add Symfony listener migration post

Very nice complex use case :)
See https://github.com/symfony/symfony/pull/36243

Way to narrow use case... not in the flow anymore


macOS installation instructions

Summary
Problem
On macOS, krunvm needs to be installed.
Solution
Should this be added to the installation instructions?
brew tap slp/krun
brew install krunvm

You are right; I forgot to add that. However, I plan to improve the installation experience in the near future. The build steps should really be for development only.
Thank you for checking out the project.

More notes:

I had to run sudomake build, otherwise I get the error Error: lsetxattr /Volumes/krunvm/root/vfs: permission denied when building libkrun.
I had to install cargo.

After that, it worked. 🙂

Great feedback. The example configs might not work as well. They are always outdated because I can't stop changing how they work 😅
I will be focus on fixing all of that in the next few days.


BUG I

Initialization file attached, simply simulate('tes.grmpy.ini'), estimate('test.grmpy.ini') should reproduce the error.
test.grmpy.txt
/home/peisenha/.envs/grmToolbox/lib/python3.5/site-packages/statsmodels/base/model.py:496: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
"Check mle_retvals", ConvergenceWarning)
../../grmpy/estimate/estimate_auxiliary.py:473: RuntimeWarning: invalid value encountered in double_scalars
init_dict['DIST']['all'][0] * init_dict['DIST']['all'][3])
Traceback (most recent call last):
File "run.py", line 22, in
grmpy.estimate('test.grmpy.ini')
File "../../grmpy/estimate/estimate.py", line 48, in estimate
write_descriptives(dict_, data, rslt)
File "../../grmpy/estimate/estimate_auxiliary.py", line 313, in write_descriptives
df3, df2 = simulate_estimation(init_dict, rslt, df1, True)
File "../../grmpy/estimate/estimate_auxiliary.py", line 264, in simulate_estimation
U, V = simulate_unobservables(dict_)
File "../../grmpy/simulate/simulate_auxiliary.py", line 40, in simulate_unobservables
U = np.random.multivariate_normal(np.zeros(3), cov, num_agents)
File "mtrand.pyx", line 4511, in mtrand.RandomState.multivariate_normal (numpy/random/mtrand/mtrand.c:37052)
File "/home/peisenha/.envs/grmToolbox/lib/python3.5/site-packages/scipy/linalg/decomp_svd.py", line 96, in svd
a1 = _asarray_validated(a, check_finite=check_finite)
File "/home/peisenha/.envs/grmToolbox/lib/python3.5/site-packages/scipy/_lib/_util.py", line 238, in _asarray_validated
a = toarray(a)
File "/home/peisenha/.envs/grmToolbox/lib/python3.5/site-packages/numpy/lib/function_base.py", line 1215, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs

So this one seems obvious. The estimation process couldn't work in a deterministic setting. How do you want to avoid that division by zero problem? Maybe the process should start from other values automatically if it is fed with deterministic distributional characteristis?

Thank you for the update. How about the following:

We throw a UserError if an estimation is requested  for the
deterministic case.
We add a flag to to the random dict generation, is_estimation, that if
true rules out deterministic models?

I think this is how I did it in the past. Let me know what you think ...
Best, Philipp
On Tue, Oct 17, 2017 at 9:50 AM, Sebastian Becker<EMAIL_ADDRESS>wrote:

So this one seems obvious. The estimation process couldn't work in a
deterministic setting. How do you want to avoid that division by zero
problem? Maybe the process should start from other values automatically if
it is fed with deterministic distributional characteristis?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/grmToolbox/grmpy/issues/79#issuecomment-337147869,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADcMJ91zTZJpIz4j9JhpXEHfmQRMttUoks5stFxDgaJpZM4P7qen
.

--
Philipp Eisenhauer
Economist
Mail<EMAIL_ADDRESS>Web www.eisenhauer.io
Repository https://github.com/peisenha

Why should we implement an is_estimation flag if we can use a constraint dictionary with probability=0.0 instead?

Yes, that is the better solution ....
On Tue, Oct 17, 2017 at 11:09 AM, Sebastian Becker<EMAIL_ADDRESS>
wrote:

Why should we implement an is_estimation flag if we can use a constraint
dictionary with probability=0.0 instead?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/grmToolbox/grmpy/issues/79#issuecomment-337168382,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADcMJ9D9Aj3lgstAJs2_-bWxhySujBhSks5stG6xgaJpZM4P7qen
.

--
Philipp Eisenhauer
Economist
Mail<EMAIL_ADDRESS>Web www.eisenhauer.io
Repository https://github.com/peisenha

I implemented a function that checks the init file for deterministic specifications during the estimation process.


Simple dependencies

Fix #84

This change is

Review status: 1 of 2 files reviewed at latest revision, 1 unresolved discussion.

bower.json, line 9 at r1 (raw file):
Previously, web-padawan (Sergey Kulikov) wrote…
the latest one is 10.0.0-beta9 (tagged but with no release notes)

thanks. Done

Comments from Reviewable

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Reviewed 1 of 2 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved, all commit checks successful.

Comments from Reviewable

Review status: all files reviewed at latest revision, all discussions resolved, all commit checks successful.

a discussion (no related file):
:confused:

Fixed -betaX dependency number requires maintenance bumps.
Some elements are installed but never used, which is a bad practice. We probably shouldn’t advocate that.

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

a discussion (no related file):
Previously, platosha (Anton Platonov) wrote…
:confused:

Fixed -betaX dependency number requires maintenance bumps.
Some elements are installed but never used, which is a bad practice. We probably shouldn’t advocate that.

Already raised that question in the issue targeting by this PR.
I also stand for using only the stuff we really need.

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

a discussion (no related file):
Previously, web-padawan (Sergey Kulikov) wrote…
Already raised that question in the issue targeting by this PR.
I also stand for using only the stuff we really need.

We are encouraging to use vaadin-core in all other starters like base-starter-vue, bakery, etc all of them depends on vaadin-core-betaX and they are updated in each new beta version.  I don't see any reason for this not be aligned with the rest.
Depending on more stuff does not necessary mean to produce bigger bundle or performance issues when running the app in dev or production, it's just a nit when running bower install which is compensated with the simply setup of the app.
In my case bower i does not produces a significant problem in a 4G network
real	0m17.140s
user	0m7.403s
sys	0m3.661s

real	0m22.917s
user	0m11.898s
sys	0m6.105s

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

a discussion (no related file):
Previously, manolo (Manuel Carrasco Moñino) wrote…
We are encouraging to use vaadin-core in all other starters like base-starter-vue, bakery, etc all of them depends on vaadin-core-betaX and they are updated in each new beta version.  I don't see any reason for this not be aligned with the rest.
Depending on more stuff does not necessary mean to produce bigger bundle or performance issues when running the app in dev or production, it's just a nit when running bower install which is compensated with the simply setup of the app.
In my case bower i does not produces a significant problem in a 4G network
real	0m17.140s
user	0m7.403s
sys	0m3.661s

real	0m22.917s
user	0m11.898s
sys	0m6.105s

OK not a blocker to me, let's align with the other starters.

Comments from Reviewable

Review status: all files reviewed at latest revision, 1 unresolved discussion.

a discussion (no related file):
Previously, web-padawan (Sergey Kulikov) wrote…
OK not a blocker to me, let's align with the other starters.

OK :see_no_evil:

Comments from Reviewable


Use this project with QUdpSocket

This is really a nice project, although I couldn't find an example specific to my problem (may be its there).
I wish to stream my audio over UDP using QUdpSocket. The connection is established and thus I just want to pass the Voice Over Udp. How can I implement this using your project. It would be much better if this can do 2 way communication (Record from Mic and play incoming voice). Thanks

Hi! I'm working on a new version where UDP will be supported, and others improvements. Currently, UDP is not supported and maybe too hard to change using the current source code.
However, it may take some time!
Thank you for your feedback!


Query server frequently offline

Hey! First of all, thanks to all of the contributors for the work done on this project. It's been extremely helpful and has saved me quite a lot of time.
I'm having an issue with the status commands through mscs. My server has two worlds running on non-standard ports. They both work just fine, and can be queried using an online tool with the expected response. When I try to query using mscs status, however, I tend to get this after the server has been running for about an hour (with a couple users online):
$ mscs status
Minecraft Server Status:
  wookies: running (query server offline).
    Process ID: 17617.
    Memory used: 1338836 kB.
  wookiescreative: running version 1.12 (0 of 20 users online).
    Process ID: 17816.
    Memory used: 1260216 kB.

Any idea what's causing this?

Hi @jwilling.  I'm not sure what is going on here.  It sounds like you have tried everything I would recommend.
Here is how it works:
The script starts a tail process (line 1595) that pipes anything that ends up in the query.in file to the Query server via socat. Responses from the Query server via socat are piped into the query.out file.  If the tail or socat process have issues for some reason, the Query server will be listed as offline.  Perhaps you have a permission issue on one of the two in/out files?  I don't know.

Hey @jwilling,

Are you having issues using mscs status with only the wookies world? Or does it occur to other worlds as well? Perhaps try creating another world and letting it run for an hour, and then see if the issue also occurs when you try to get the status of that world; this could help tell us if it's world-specific or if there's something else going on. The second thing I would like to check is whether or not the amount of users online has something affecting it; did you find that mscs status works when, say, there's less than 3 users online and then stops working when there's more than that?  Perhaps the issue centers around how many users are online.

Also, are you running MSCS in a multi-user environment / what user(s) are you using to running MSCS? (That is, if you didn't follow the default installation). It could be a permission issue as @sandain noted

Sorry that we can't give you a clear cut answer; right now we have to isolate the problem and then figure it out at this point.

Huge thanks @sandain and @Roflicide for your assistance.
So my original use case for needing the status command to function was for a status page. I'd gather the player data from the query-json command and pipe it into a templated page for checking who was online. This job ran every minute. I'm sure there are better ways to do this, but I just wanted something quick and dirty.
Since I was having issues with the reliability of the variants of status, I switched to Dinnerbone/mcstatus to get the query information I needed. Interestingly, ever since I switched to using mcstatus, the status query in mscs functions as expected.
It definitely could be something related to user permissions, although as far as I can remember I ran everything from the same user account. I definitely did do a custom installation, as I wanted everything contained within my home directory, in addition to having a different username. And yes, it only occurred with one specific world.
If this ever happens again I'll try to figure out what variables might be at play. That being said, it seems like I'm the only one that had this problem and my use cases are fairly specific, so we could close this and I could reopen it if it occurs later, if you'd prefer.

I had this issue too, but everytime this occured I ran a second query and it came back normal so I didn't evaluate what goes wrong.
This happend more often on modded servers running with Forge, vanilla-server querys where as long as I remember fine.
If this occures again I will report back.

If you notice, the code in the queryStart method that I linked to above includes a 20 second delay before starting the Query handler code (that uses tail and socat). This means that for the first 20 seconds after a server starts up, the expected response is _query server offline_. If you are still getting that response after 20 seconds infrequently there is probably no issue -- if the querySendPacket` fails to decode a response from the server, it won't try again unless you ask it and this happening infrequently should be no problem.  If this is happening more often than infrequently, then there is an issue.

@conrad784 thank you for the input, you are right when it seems to be happening to forge servers.
@sandain I have confirmed that this seems to be a bug on forge servers. To reproduce:

Create a new forge server
Start it
Run a mscs status command

The first time you run the status, it will list the query server as offline. The second time you run it, the command works and it appears to correctly query it.
This issue seems to occur every time after stopping/starting the forge world, and it seems to fix itself after the first query.

I'm going to go ahead and mark this bug as invalid and close it.  If anyone is able to come up with a condition where the  "query server offline" message is received after the initial 20 second startup time, please open a new bug.

Having this issue as well, even after the first 20 seconds. What do you need from me to figure this out?

@Aersaud
Are you running a vanilla server or with mods? What version of minecraft? Does it happen to all of your worlds? Did you follow the default installation process? Is there any output in the console?

Running with mods, forge and sponge. Recommended version for both of those. Also a few other mods, if you like I can name them. However, when testing this I ran the server modless, a side from forge and sponge. We only have 1 world so I cannot check any others unfortunately. Followed the default install, including server backups. When this happens the server works fine and there are no errors in either the linux console or the game console.  It also ends up resolving and fixing itself after some time (I've seen about 5-10 minutes) sometimes it never resolves and I have to keep starting and stopping the server for it to eventually work. I am checking it using the mscs status [world] command.

Please post a copy of your latest.log, and the query.in and query.out files while you are experiencing this issue. I'll see if I can track it down.

Ok so while my server was running the query server went offline. I cp'ing the files now to SFTP them over here and post.

Well that didn't really help. The latest.log shows that the query server is running and I see no error messages that would note the server is having issues with query.
[10:16:35] [Query Listener #1/INFO]: Query running on <IP_ADDRESS>:25565

I am curious what this error message means:
java.io.EOFException: null
	at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268) ~[?:1.8.0_162]
	at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258) ~[?:1.8.0_162]
	at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164) ~[?:1.8.0_162]
	at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79) ~[?:1.8.0_162]
	at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91) ~[?:1.8.0_162]
	at net.minecraft.nbt.CompressedStreamTools.func_74796_a(CompressedStreamTools.java:26) ~[gi.class:?]
	at net.minecraft.world.storage.SaveFormatOld.loadAndFix(SaveFormatOld.java:117) [bfc.class:?]
	at net.minecraft.world.storage.SaveHandler.redirect$onLoadWorldInfo$bbi000(SaveHandler.java:1068) [bfb.class:?]
	at net.minecraft.world.storage.SaveHandler.func_75757_d(SaveHandler.java:122) [bfb.class:?]
	at org.spongepowered.common.world.WorldManager.loadAllWorlds(WorldManager.java:706) [WorldManager.class:1.12.2-2611-8.0.0-BETA-3018]
	at net.minecraft.server.MinecraftServer.func_71247_a(MinecraftServer.java:3530) [MinecraftServer.class:?]
	at net.minecraft.server.dedicated.DedicatedServer.func_71197_b(DedicatedServer.java:270) [nz.class:?]
	at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:484) [MinecraftServer.class:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]

It doesn't look like it should break query, but maybe you should look into what is causing that.
The query.in file looks correct, but the query.out file is empty meaning commands were being sent but no response was received for some reason. Check which processes are currently running when you get this problem (run ps -A). You should have a couple tail processes and a socat process running -- tail is used to watch for changes to the query.in file and pipe it to socat, and socat sends the query to the server and pipes the response to the query.out file.

mcadmin@tkmminecraft:~$ ps -A
PID TTY          TIME CMD
1 ?        00:00:03 systemd
2 ?        00:00:00 kthreadd
3 ?        00:00:00 ksoftirqd/0
5 ?        00:00:00 kworker/0:0H
7 ?        00:00:21 rcu_sched
8 ?        00:00:00 rcu_bh
9 ?        00:00:00 migration/0
10 ?        00:00:01 watchdog/0
11 ?        00:00:01 watchdog/1
12 ?        00:00:00 migration/1
13 ?        00:00:01 ksoftirqd/1
15 ?        00:00:00 kworker/1:0H
16 ?        00:00:01 watchdog/2
17 ?        00:00:00 migration/2
18 ?        00:00:00 ksoftirqd/2
20 ?        00:00:00 kworker/2:0H
21 ?        00:00:01 watchdog/3
22 ?        00:00:00 migration/3
23 ?        00:00:18 ksoftirqd/3
25 ?        00:00:00 kworker/3:0H
26 ?        00:00:00 kdevtmpfs
27 ?        00:00:00 netns
28 ?        00:00:00 perf
29 ?        00:00:00 khungtaskd
30 ?        00:00:00 writeback
31 ?        00:00:00 ksmd
32 ?        00:00:03 khugepaged
33 ?        00:00:00 crypto
34 ?        00:00:00 kintegrityd
35 ?        00:00:00 bioset
36 ?        00:00:00 kblockd
37 ?        00:00:00 ata_sff
38 ?        00:00:00 md
39 ?        00:00:00 devfreq_wq
44 ?        00:00:00 kswapd0
45 ?        00:00:00 vmstat
46 ?        00:00:00 fsnotify_mark
47 ?        00:00:00 ecryptfs-kthrea
63 ?        00:00:00 kthrotld
64 ?        00:00:00 acpi_thermal_pm
65 ?        00:00:00 bioset
66 ?        00:00:00 bioset
67 ?        00:00:00 bioset
68 ?        00:00:00 bioset
69 ?        00:00:00 bioset
70 ?        00:00:00 bioset
71 ?        00:00:00 bioset
72 ?        00:00:00 bioset
73 ?        00:00:00 scsi_eh_0
74 ?        00:00:00 scsi_tmf_0
75 ?        00:00:00 scsi_eh_1
76 ?        00:00:00 scsi_tmf_1
82 ?        00:00:00 ipv6_addrconf
95 ?        00:00:00 deferwq
96 ?        00:00:00 charger_manager
97 ?        00:00:00 bioset
151 ?        00:00:00 kpsmoused
153 ?        00:00:00 scsi_eh_2
155 ?        00:00:00 scsi_tmf_2
192 ?        00:00:00 kworker/0:1H
194 ?        00:00:00 bioset
269 ?        00:00:00 raid5wq
293 ?        00:00:00 kdmflush
295 ?        00:00:00 bioset
304 ?        00:00:00 kdmflush
305 ?        00:00:00 bioset
317 ?        00:00:00 bioset
342 ?        00:00:07 jbd2/dm-0-8
343 ?        00:00:00 ext4-rsv-conver
400 ?        00:00:00 kworker/2:1H
408 ?        00:00:00 iscsi_eh
413 ?        00:00:00 systemd-journal
420 ?        00:00:00 ib_addr
422 ?        00:00:00 kauditd
433 ?        00:00:00 ib_mcast
435 ?        00:00:00 ib_nl_sa_wq
437 ?        00:00:00 ib_cm
441 ?        00:00:00 iw_cm_wq
442 ?        00:00:00 rdma_cm
445 ?        00:00:00 lvmetad
526 ?        00:00:03 kworker/1:1H
580 ?        00:00:00 iprt-VBoxWQueue
730 ?        00:00:00 kworker/3:1H
788 ?        00:00:00 ttm_swap
796 ?        00:00:00 ext4-rsv-conver
936 ?        00:00:00 rsyslogd
938 ?        00:00:00 lxcfs
940 ?        00:00:00 atd
942 ?        00:00:00 cron
946 ?        00:00:00 acpid
954 ?        00:00:00 dbus-daemon
975 ?        00:00:00 systemd-logind
979 ?        00:00:10 snapd
981 ?        00:00:03 accounts-daemon
992 ?        00:00:00 mdadm
1000 ?        00:00:00 polkitd
1079 ?        00:00:00 dhclient
1139 ?        00:00:00 sshd
1163 ?        00:00:07 iscsid
1164 ?        00:00:32 iscsid
1217 ?        00:00:07 irqbalance
1227 tty1     00:00:00 agetty
1528 ?        00:00:00 systemd
1531 ?        00:00:00 (sd-pam)
5579 ?        00:00:21 kworker/0:2
9818 ?        00:00:00 sh
9820 ?        00:00:02 tail
9821 ?        00:00:00 sh
9822 ?        00:33:56 java
10130 ?        00:00:01 kworker/3:0
10709 ?        00:00:11 kworker/0:0
11687 ?        00:00:00 kworker/2:1
13008 ?        00:00:00 kworker/u8:1
13012 ?        00:00:00 kworker/u8:0
13020 ?        00:00:00 kworker/u8:2
13064 ?        00:00:00 kworker/3:1
15560 ?        00:00:00 kworker/1:2
15897 ?        00:00:00 kworker/2:2
15898 ?        00:00:00 kworker/2:3
15906 ?        00:00:00 sshd
15937 ?        00:00:00 sshd
15938 pts/0    00:00:00 bash
17062 ?        00:00:00 systemd-timesyn
17064 ?        00:00:00 kworker/0:1
19820 ?        00:00:00 systemd-udevd
20357 ?        00:00:00 xfsalloc
20358 ?        00:00:00 xfs_mru_cache
20363 ?        00:00:00 jfsIO
20364 ?        00:00:00 jfsCommit
20365 ?        00:00:00 jfsCommit
20366 ?        00:00:00 jfsCommit
20367 ?        00:00:00 jfsCommit
20368 ?        00:00:00 jfsSync
20895 ?        00:00:04 kworker/1:1
21819 ?        00:00:00 kworker/2:0
25742 ?        00:00:02 kworker/1:0
28013 pts/0    00:00:00 ps
I see a tail process, but not seeing socat.
It's happening again by the way.

You only have one tail not two, and no socat.  That is definitely part of the problem. The question is why that is the case.
I noticed that error also. You might want to look into what is causing that, it is not the script, but might be behind the issue with query.

I found the error from the Sponge github. Apparently its a corrupted level.dat - I wonder if this may be causing it.

Ok just another update. Left it running all night and it did the same thing. Query server went offline randomly.

Hi again @Aersaud,
I'm sorry you're having an issue. I suspect what is going on is that socat's default timeout (0.5 s) is too low for certain server environments; however, this is just a guess. Let's see what exactly is going on. If you could, please do the following:

Stop the world.
Type which msctl to get the location of the script file. In my case it was /usr/local/bin/msctl
Edit line 1602 of that file, and replace
$SOCAT -
with
 $SOCAT -d -d -d -lf \"$WORLD_DIR/socat.log\" -

So now lines 1599-1603 should look like this:
nohup sh -c "
  sleep 20;
  tail -f --pid=$SERVER_PID \"$WORLD_DIR/query.in\" |
  $SOCAT -d -d -d -lf \"$WORLD_DIR/socat.log\" - UDP:$SERVER_IP:$QUERY_PORT > \"$WORLD_DIR/query.out\"
" > /dev/null 2>&1 &

Start the world. About a minute after starting it, there should be a file created called socat.log in the world directory (/opt/mscs/worlds/<yourworld>). This will log all of the information on what socat is doing to that file.

Once you've followed the above steps, please execute the status queries as you normally would. Once the issue comes, paste the contents of the socat.log file here so we can see why exactly socat is terminating.
Thanks!

Thanks for the help @Roflicide, I wasn't sure how to debug this issue further.

This is from the latest time it went down.

2018/04/21 23:22:42 socat[4349] I This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. $
2018/04/21 23:22:42 socat[4349] I This product includes software written by Tim Hudson<EMAIL_ADDRESS>2018/04/21 23:22:42 socat[4349] N reading from and writing to stdio
2018/04/21 23:22:42 socat[4349] N opening connection to AF=2 <IP_ADDRESS>:25566
2018/04/21 23:22:42 socat[4349] I starting connect loop
2018/04/21 23:22:42 socat[4349] I socket(2, 2, 17) -> 6
2018/04/21 23:22:42 socat[4349] N successfully connected from local address AF=2 <IP_ADDRESS>:57655
2018/04/21 23:22:42 socat[4349] I resolved and opened all sock addresses
2018/04/21 23:22:42 socat[4349] N starting data transfer loop with FDs [0,1] and [6,6]
2018/04/21 23:23:12 socat[4349] I transferred 11 bytes from 0 to 6
2018/04/21 23:23:12 socat[4349] E read(6, 0x84c100, 8192): Connection refused
2018/04/21 23:23:12 socat[4349] N exit(1)
2018/04/21 23:23:12 socat[4349] I shutdown(6, 2)

For some reason it looks like it's opening a connection to port 25566 even though your query server seems to be running on port 25565 based on the latest.log you uploaded earlier. This would explain why it's failing to read a connection.
Can you paste the contents of your server.properties file here? Specifically, I want to see what the values are (or if the values are even there!) for the properties query.port, query-enabled, and server-port.

Yes sorry forgot that I changed a few things. It is now running on port 25566. Here is the updated server.properties.

#Minecraft server properties
#Sat Apr 21 23:23:09 EDT 2018
spawn-protection=16
max-tick-time=60000
query.port=25566
generator-settings=
force-gamemode=false
allow-nether=true
gamemode=0
broadcast-console-to-ops=true
enable-query=true
player-idle-timeout=0
difficulty=2
spawn-monsters=true
op-permission-level=4
pvp=true
snooper-enabled=true
level-type=DEFAULT
hardcore=false
enable-command-block=false
max-players=25
network-compression-threshold=256
resource-pack-sha1=
max-world-size=29999984
rcon.port=25575
server-port=25566
server-ip=
spawn-npcs=true
allow-flight=false
level-name=TKM-MadCraft
view-distance=10
resource-pack=
spawn-animals=true
white-list=false
rcon.password=CHANGEDFORPRIVACY
generate-structures=true
max-build-height=256
online-mode=false
level-seed=
prevent-proxy-connections=false
use-native-transport=true
motd="Welcome to TKM-MadCraft\!"
enable-rcon=true

Let's try another modification to the script in the same place. Add the -s option to socat to make it 'sloppy with errors'.
This should make lines 1599-1603 look like:
nohup sh -c "
  sleep 20;
  tail -f --pid=$SERVER_PID \"$WORLD_DIR/query.in\" |
  $SOCAT -d -d -d -lf \"$WORLD_DIR/socat.log\" -s - UDP:$SERVER_IP:$QUERY_PORT > \"$WORLD_DIR/query.out\"
" > /dev/null 2>&1 &

A more complicated option would be to explore the -W option. This would require wrapping the code to start the query handler in a while loop that checks for the existence of the logfile declared with the -W option and the current state of the server. If the server is running, but the log file is missing, the query handler would be restarted.  This option requires some coding, so lets give the simple 'sloppy' option above a try first.

Just updated my msctl file using nano. It is now displaying
nohup sh -c "
    sleep 20;
    tail -f --pid=$SERVER_PID \"$WORLD_DIR/query.in\" |
    $SOCAT -d -d -d -lf \"$WORLD_DIR/socat.log\" -s - UDP:$SERVER_IP:$QUERY_PORT > \"$WORLD_DIR/query.out\"
  " > /dev/null 2>&1 &

Whats next?

Restart your server with that change in place and let it run until query ends up failing (or not). Post the socat.log if it fails.

2018/04/22 21:46:48 socat[15897] N opening connection to AF=2 <IP_ADDRESS>:25566
2018/04/22 21:46:48 socat[15897] I starting connect loop
2018/04/22 21:46:48 socat[15897] I socket(2, 2, 17) -> 6
2018/04/22 21:46:48 socat[15897] N successfully connected from local address AF=2 <IP_ADDRESS>:58939
2018/04/22 21:46:48 socat[15897] I resolved and opened all sock addresses
2018/04/22 21:46:48 socat[15897] N starting data transfer loop with FDs [0,1] and [6,6]
2018/04/22 21:47:05 socat[15897] I transferred 11 bytes from 0 to 6
2018/04/22 21:47:05 socat[15897] E read(6, 0x1db8100, 8192): Connection refused
2018/04/22 21:47:05 socat[15897] N socket 2 to socket 1 is in error
2018/04/22 21:47:05 socat[15897] N socket 2 (fd 6) is at EOF
2018/04/22 21:47:05 socat[15897] I poll timed out (no data within 0.500000 seconds)
2018/04/22 21:47:05 socat[15897] I shutdown(6, 2)
2018/04/22 21:47:05 socat[15897] N exiting with status 0

@sandain
Latest from the socat.log when I ran mscs status

Well it came back up on it's own without me doing anything currently......I will check that website out to query it when it tells me it is down.
Here is the last bit of it going down and up..
I check the status using mscs status every so often to make sure.
2018/04/22 22:54:36 socat[20356] N opening connection to AF=2 <IP_ADDRESS>:25566
2018/04/22 22:54:36 socat[20356] I starting connect loop
2018/04/22 22:54:36 socat[20356] I socket(2, 2, 17) -> 6
2018/04/22 22:54:36 socat[20356] N successfully connected from local address AF=2 <IP_ADDRESS>:60974
2018/04/22 22:54:36 socat[20356] I resolved and opened all sock addresses
2018/04/22 22:54:36 socat[20356] N starting data transfer loop with FDs [0,1] and [6,6]
2018/04/22 22:55:41 socat[20356] N socket 1 (fd 0) is at EOF
2018/04/22 22:55:41 socat[20356] I shutdown(6, 1)
2018/04/22 22:55:42 socat[20356] I poll timed out (no data within 0.500000 seconds)
2018/04/22 22:55:42 socat[20356] I shutdown(6, 2)
2018/04/22 22:55:42 socat[20356] N exiting with status 0
2018/04/22 22:57:40 socat[22210] I socat by Gerhard Rieger - see www.dest-unreach.org
2018/04/22 22:57:40 socat[22210] I This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.o$
2018/04/22 22:57:40 socat[22210] I This product includes software written by Tim Hudson<EMAIL_ADDRESS>2018/04/22 22:57:40 socat[22210] N reading from and writing to stdio
2018/04/22 22:57:40 socat[22210] N opening connection to AF=2 <IP_ADDRESS>:25566
2018/04/22 22:57:40 socat[22210] I starting connect loop
2018/04/22 22:57:40 socat[22210] I socket(2, 2, 17) -> 6
2018/04/22 22:57:40 socat[22210] N successfully connected from local address AF=2 <IP_ADDRESS>:48128
2018/04/22 22:57:40 socat[22210] I resolved and opened all sock addresses
2018/04/22 22:57:40 socat[22210] N starting data transfer loop with FDs [0,1] and [6,6]
2018/04/23 14:59:08 socat[22210] I transferred 11 bytes from 0 to 6
2018/04/23 14:59:08 socat[22210] I transferred 13 bytes from 6 to 1
2018/04/23 14:59:09 socat[22210] I transferred 15 bytes from 0 to 6
2018/04/23 14:59:09 socat[22210] I transferred 195 bytes from 6 to 1
2018/04/23 16:12:06 socat[22210] I transferred 11 bytes from 0 to 6
2018/04/23 16:12:06 socat[22210] I transferred 13 bytes from 6 to 1
2018/04/23 16:12:07 socat[22210] I transferred 15 bytes from 0 to 6
2018/04/23 16:12:07 socat[22210] I transferred 202 bytes from 6 to 1
2018/04/23 16:27:01 socat[22210] I transferred 11 bytes from 0 to 6
2018/04/23 16:27:01 socat[22210] I transferred 14 bytes from 6 to 1
2018/04/23 16:27:02 socat[22210] I transferred 15 bytes from 0 to 6
2018/04/23 16:27:02 socat[22210] I transferred 202 bytes from 6 to 1
2018/04/23 16:27:10 socat[22210] I transferred 11 bytes from 0 to 6
2018/04/23 16:27:10 socat[22210] I transferred 14 bytes from 6 to 1
2018/04/23 16:27:11 socat[22210] I transferred 15 bytes from 0 to 6
2018/04/23 16:27:11 socat[22210] I transferred 202 bytes from 6 to 1
2018/04/23 16:28:59 socat[22210] I transferred 11 bytes from 0 to 6
2018/04/23 16:28:59 socat[22210] I transferred 13 bytes from 6 to 1
2018/04/23 16:29:00 socat[22210] I transferred 15 bytes from 0 to 6
2018/04/23 16:29:00 socat[22210] I transferred 202 bytes from 6 to 1
2018/04/23 16:29:51 socat[22210] I transferred 11 bytes from 0 to 6
2018/04/23 16:29:51 socat[22210] I transferred 14 bytes from 6 to 1
2018/04/23 16:29:52 socat[22210] I transferred 15 bytes from 0 to 6
2018/04/23 16:29:52 socat[22210] I transferred 202 bytes from 6 to 1

I have a feeling the 2018/04/22 22:55:42 socat[20356] I poll timed out (no data within 0.500000 seconds) kind of points to our issue as you described earlier where it is too short of a time frame.
@sandain

I think with PR #242 by @Roflicide, we can finally put this issue to rest.  Feel free to reopen this issue if you are still affected.


GitOps Progressive Delivery

Use case:
as a platform team,
I want different deployment strategies via traffic routing
so that I can increase observability of large scale rollouts of user-facing applications
Possible solution: using Flagger within a GitOps setup based on Flux

a subset use cases of this larger epic is this internal use case/epic: https://github.com/giantswarm/roadmap/issues/620

@mproffitt is this issue still valid?

@weatherhog  We discussed Flagger recently and potentially moving this topic to cabbage. Is there an issue there for it? if so, I'd be in favour linking it here and then closing this ticket.

closing this for https://github.com/giantswarm/roadmap/issues/3684


Automator: update common-files@master in istio/istio@master

Generated by Automator - 2022-03-15T17:57:03+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
2ef46e5657c25199709440f125f3c0732b655085
link
unknown
/test gencheck_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test gencheck_istio


Write service administration manual

Placeholder issue for writing the systems administration manual for this. cc @NerdyGirl537

What format will this take?

Made an initial brain dump in the google drive, will follow up with @NerdyGirl537 after deployment.

I did this and turned over to the DCAF board, I think we can call this resolved for all intents and purposes. Marking closed.


Update configuration after variables has been injected to the main configuration

Just adds configuration update after injecting variables to main.cf.
fix #48

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: tamaro-skaljic:x: cenk1cenk2You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you very much for the contribution!


Support Parquet Benchmark for BigInt and Double with different Filter and Null rates

This PR supports Parquet Benchmark for BigInt and Double with different Filter and Null rates on both Dictionary and Plain encoded Parquet files. Including:
Refactored the initial parquet benchmark so that it is able to support the benchmark on different types, filter rates, null rates and both Dictionary/Plain Parquet file encoding types.
Added the the element counter to the filter generator and increased the sample size to 65536 for more accurate filter rate estimation. Also, Added allowNulls to FilterSpec to generator non-null filters for the benchmark.

Hi @liushengxuan , Can you resolve conflicts ?
cc : @Yuhta

Duplicate of #3211


refactor(icons)!: Use a better logic to parse icons.

[View Diff Here] (Compared to #285).
This commit refactored the icon parsing logic. See the code for implementation detail. @CharlesChiuGit Using the get function  might be a better option, suggestions for improvement are welcome 👍
THIS COMMIT MAY REQUIRE CHECKS & REVIEWS BEFORE MERGING. Though I've doubled-checked the code, there's still possibility that I missed some changes.

Wow, this looks way nicer! Thanks for refactoring it!

I think the use_icon flag is misunderstanding. Icons in lualine and nvim-tree and telescope's candidate files are still here though I have set it to false. So maybe you need to config for every corner using icons to keep it intuitive and coherent.

Additionally, there is a inconspicuous bug in telescope.

It can be reproduced by <C-p> and <C-n> repeatly in telescope.

In fact, I think configurable use_icons is not a good idea because it hurts the complete user experience.

Indeed. I have also fixed relevant issues, now Telescope should work correctly:
https://user-images.githubusercontent.com/50296129/196016704-e33c3680-b5a6-4586-a12b-24d726b281d2.mp4


nim is very slow on first boot due to persistent subscriptions

The recently added persistent subscriptions in nim for ControllerCert and CipherContext makes nim wait 'forever' when /persist is blank (such as make run, or installing a new device).
We haven't used persistent subscriptions before; the intent was to read initial content from directory if it exists then use IPC.

Ah, is this what caused the problem? We started reading persistent, which set subFromDir = true, which means the returned &Subscriber{subscribeFromDir: true}, which is only ever used [here[(https://github.com/lf-edge/eve/blob/f1c08de370f7302aeb6cb24d9a2e4e1bddac34fb/pkg/pillar/pubsub/socketdriver/subscribe.go#L83):
	if s.subscribeFromDir {
		// Waiting for directory to appear
		for {
			if _, err := os.Stat(s.dirName); err != nil {
				errStr := fmt.Sprintf("Subscribe(%s): failed %s; waiting", s.name, err)
				log.Errorln(errStr)
				time.Sleep(10 * time.Second)
			} else {
				break
			}
		}
		// WatchStatus updates the channel with messages in a slightly different
		// format than the standard for pubsub
		// we pass it through a translator
		translator := make(chan string)
		go watch.WatchStatus(s.dirName, true, translator)
		go s.translate(translator, s.C)
		return nil
	} else if subscribeFromSock {
		go s.watchSock()
		return nil
	} else {
		errStr := fmt.Sprintf("Subscribe(%s): failed %s", s.name, "nowhere to subscribe")
		return errors.New(errStr)
	}

and so it waits forever for the directory to appear.
How do we get the load to happen? Is that already taken care of?
So the flow would look like this:

Create subscription
Load from dir:

exists: read it
does not exist: ignore it

Subscriber: read from IPC

Is that correct?

So the flow would look like this:

Create subscription

Load from dir:

exists: read it
does not exist: ignore it

Subscriber: read from IPC

Is that correct?

Yes, that was the intent when I added the Load. But the subscriber waiting for the directory was a surprise to me. Wasn't seen in testing because even qemu comes up; just takes 5-10 minutes longer .

Wouldn't this break publishing? Any previously persistent reads not only initialized from the dir, but also subscribed via watching files:
		go watch.WatchStatus(s.dirName, true, translator)
		go s.translate(translator, s.C)

which is replaced (when subFromDir == false) with this:
		go s.watchSock()

The publish is here:
func (s *Publisher) Publish(key string, item []byte) error {
	fileName := s.dirName + "/" + key + ".json"
	log.Debugf("Publish writing %s\n", fileName)

	err := fileutils.WriteRename(fileName, item)
	if err != nil {
		return err
	}
	return nil
}

That doesn't appear to publish to the socket at all. Or did I miss it?

FWIW, I did couple of tests on these changes:

For new installations:
I deleted persist/status/zedagent directory and did device reboot. After reboot, nim came up successfully.
For already running devices:
Wifi creds are decrypting successfully at boot up.

That doesn't appear to publish to the socket at all. Or did I miss it?

@deitch what you quoted is just the socketdriver Publish.
The real Publish doing the bulk of the work is
func (pub *PublicationImpl) Publish(key string, item interface{}) error {

what you quoted is just the socketdriver Publish

OK. I did the work teasing it apart, but I really don't remember.

As long as it passes all your tests, I am not going to complain!


.circleci folder

Hi, I noticed when using this package that the circletron config is required to go in a folder .circle.
The other configuration for Circle as outlined in their docs goes in .circleci: https://circleci.com/docs/2.0/configuration-reference/
So now we have two folders in our repo, .circle (for circletron) and .circleci (for everything else).
It's not an urgent issue but would be nice to keep it consistent. Our first implementation put the config file in the wrong folder because it's easy to miss.

Ah yes I should have chosen .circleci from the beginning. I'll update it to also load from this location and use .circle as a fallback for those already using configurations.
I would have noticed earlier but in all my projects I'm using the default configuration so don't need a configuration file.

I think there may have been some confusion here. The source files are looking for .circleci/circletron.yml but the readme says to put that config in .circle/circletron.yml. I'll open a PR to update the readme so it matches what the source code expects.
https://github.com/circletron/circletron/blob/main/src/index.ts#L223

@sethmcleod Thanks for the PR :)


Respect --pre option in requirements.in file

Fixes #819.
Changelog-friendly one-liner: Respect --pre option in a requirements.in file
Contributor checklist

[X] Provided the tests for the changes.
[x] Requested a review from another contributor.
[X] Gave a clear one-line description in the PR (that the maintainers can add to CHANGELOG.md on release).
[x] Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

pip-tools v3.8.0 is released


x/website:Syntax highlighting in examples

just add this simple pice of code into the site footer template (i believe that it is just one file):
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/10.5.0/styles/darcula.min.css" />

<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/10.5.0/highlight.min.js"></script>
<script>
  document.querySelectorAll('pre')
    .forEach(function(el) {
      hljs.highlightBlock(el);
    })
</script>

and all the documentation will looks like:

instead of old ugly plain <pre> formatting:

you have to add only that little pice of code and we'll get pretty highlighted docs!

Duplicate of #13178


ldflags as an Option

This is the way that I did it for a "quick fix." I am unsure if the way I did it is the best way. Hopefully this helps :D

Actually, this looks like a pretty reasonable patch. The only thing: can you add the option to set ldflags in Arm7Binary.__init__?

I added the options in Arm7Binary.init and I think I did the git squash. First time doing it so let me know if it should be done differently.

Actually, can you rebase it? I usually prefer to merge commits without any changes so that they appear with their original author:
git fetch --all
git rebase origin/master
git push [myfork] ldflags -f

The commit is fine, though, as soon as this is done I'll apply the patch.

Nevermind, after a quick chat in Discord I've just rebased the branch myself and pushed the changes: https://github.com/AntonioND/architectds/commit/f04dafa739837bd0336bf208a627ca165d1c73bf