Update: Improve query name override to work with group labels

What does this PR do?
Updates the plugin to improve custom display names for queries, including:

Fix a bug where setting a query name caused the group by labels to dissapear
Update the input name and description to be more accurate

Related
Closes #7
PRs #8 and #10 are related to this issue - The update pulls the proposed update in #8 into the existing broken Legend configuration to keep the query options as concise as possible.
#10 involves breaking changes to current formats and needs further product discussion before it's ready.
Why are you doing it?
Currently if you try to use the Legend configuration for a query, the results are inconsistent and disappointing. If your query has group by values then the Legend name will cause those labels to disappear:

This update fixes that issue so that custom query names work consistently for queries with and without group by labels:

@kostz thanks again for all your contributions on this issue 🏆
Because there's a number of potentially impacting changes to work through switching to the native Grafana implementation of labels (proposal in #9), product has recommended we move forward with your fix from #8 for the currently broken Legend field.
Hopefully this will enable the UX you're looking for with a single query config option 👍

This looks great to me! Thanks for finding this excellent compromise/enhancement to the plugin @DHedgecock.

@kostz thanks again for all your contributions on this issue 🏆
Because there's a number of potentially impacting changes to work through switching to the native Grafana implementation of labels (proposal in #9), product has recommended we move forward with your fix from #8 for the currently broken Legend field.
Hopefully this will enable the UX you're looking for with a single query config option 👍

Hey @DHedgecock , I'm curious, you said you'll go with my pull request, but here merged yours, please advise the background of this decision

@kostz thanks again for all your contributions on this issue 🏆
Because there's a number of potentially impacting changes to work through switching to the native Grafana implementation of labels (proposal in #9), product has recommended we move forward with your fix from #8 for the currently broken Legend field.
Hopefully this will enable the UX you're looking for with a single query config option 👍

Hey @DHedgecock , I'm curious, you said you'll go with my pull request, but here merged yours, please advise the background of this decision

Hey @kostz - apologies for not fully communicating the update.
We synced with product on the changes and wanted to enable setting a custom query name using a single query option to try and keep the UX as intuitive as possible.
I wanted to be respectful of your time so I made the update rather than asking you to go back and re-work your PR.
Thanks for the feedback that this was surprising, we definitely value your feedback and the work you contributed 👍


Change to 'ruby-fifo' from 'fifo'

'fifo' will be seen as pointing to the https://github.com/sidbatra/fifo on https://rubygems.org.

I tried to use your plug from docker contaier. However, the following error message is output.
root@a314c02dae58:/# bundle install
...
Bundle complete! 4 Gemfile dependencies, 18 gems now installed.
Bundled gems are installed into .usr/local/bundle.

root@a314c02dae58:/# fluentd
2015-11-27 05:51:06 +0000 [info]: reading config file path="/etc/fluent/fluent.conf"
2015-11-27 05:51:06 +0000 [info]: starting fluentd-0.12.17
2015-11-27 05:51:06 +0000 [info]: gem 'fluentd' version '0.12.17'
2015-11-27 05:51:06 +0000 [info]: gem 'fluent-plugin-cloudwatch-logs' version '0.1.2'
2015-11-27 05:51:06 +0000 [info]: gem 'fluent-plugin-named_pipe' version '0.1.1'
2015-11-27 05:51:06 +0000 [info]: adding match pattern="foo.bar.**" type="named_pipe"
/usr/local/bundle/gems/fluent-plugin-named_pipe-0.1.1/lib/fluent/plugin/out_named_pipe.rb:13:in `require': cannot load such file -- fifo (LoadError)
        from /usr/local/bundle/gems/fluent-plugin-named_pipe-0.1.1/lib/fluent/plugin/out_named_pipe.rb:13:in `initialize'
        from /usr/local/bundle/gems/fluentd-0.12.17/lib/fluent/plugin.rb:128:in `new'
...

Used docker image is 'ruby:2.2.3'.
Gemfile
source "https://rubygems.org"

gem "ruby-fifo", :git => "https://github.com/shurizzle/ruby-fifo"
gem "fluentd"
gem "fluent-plugin-named_pipe"
gem "fluent-plugin-cloudwatch-logs"

fluent.conf
<source>
  type forward
</source>

<source>
  type   named_pipe
  tag    cwl.test
  path   /path/to/file
  format none
</source>

<match foo.bar.**>
  type   named_pipe
  path   /path/to/file
</match>

<match cwl.test>
  type               cloudwatch_logs
  log_group_name     aws/ec2/cwl-test
  auto_create_stream true
  use_tag_as_stream  true
</match>

It looks to move After replacement.
root@a314c02dae58:/# sed -i -e "s/require 'fifo'/require 'ruby-fifo'/g" /usr/local/bundle/gems/fluent-plugin-named_pipe-0.1.1/lib/fluent/plugin/*.rb
root@a314c02dae58:/# fluentd
2015-11-27 06:17:41 +0000 [info]: reading config file path="/etc/fluent/fluent.conf"
2015-11-27 06:17:41 +0000 [info]: starting fluentd-0.12.17
2015-11-27 06:17:41 +0000 [info]: gem 'fluentd' version '0.12.17'
2015-11-27 06:17:41 +0000 [info]: gem 'fluent-plugin-cloudwatch-logs' version '0.1.2'
2015-11-27 06:17:41 +0000 [info]: gem 'fluent-plugin-named_pipe' version '0.1.1'
2015-11-27 06:17:41 +0000 [info]: adding match pattern="foo.bar.**" type="named_pipe"
2015-11-27 06:17:41 +0000 [info]: adding match pattern="cwl.test" type="cloudwatch_logs"
2015-11-27 06:17:41 +0000 [info]: adding source type="forward"
2015-11-27 06:17:41 +0000 [info]: adding source type="named_pipe"
2015-11-27 06:17:41 +0000 [info]: using configuration file: <ROOT>
  <source>
    type forward
  </source>
  <source>
    type named_pipe
    tag cwl.test
    path /path/to/file
    format none
  </source>
  <match foo.bar.**>
    type named_pipe
    path /path/to/file
  </match>
  <match cwl.test>
    type cloudwatch_logs
    log_group_name aws/ec2/cwl-test
    auto_create_stream true
    use_tag_as_stream true
  </match>
</ROOT>
2015-11-27 06:17:41 +0000 [info]: listening fluent socket on <IP_ADDRESS>:24224

私の環境でのみ起きてることなのかもしれませんが、、

It looks the library code base was completely changed on from 0.0.1 to 0.1.0 (it was like this on 0.0.1  > https://github.com/shurizzle/ruby-fifo/commit/6c4d4dd2cf02d5ed6cb28283b33909052ff04250)

Test is failing because ruby-fifo is broken.

Issued https://github.com/shurizzle/ruby-fifo/issues/8

@mijime you may write your Gemfile as below for now
gem 'ruby-fifo', '0.0.1'

Thanks for reply and send issue ! I've confirmed, just fixed the gem version.


pull request for e2e test at 01/24/2017 23:26:09

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


binance_client.get_all_tickers() gets "RuntimeError: maximum recursion depth exceeded while calling a Python object" after 300+ calls

Hi,
Firstly thank you for your work :)
I have written a simple procedural Python script with utilizes your module, but I'm getting the following assert from get_all_tickers() once it has been run at least 300+ times;
  File "./__main__.py", line 86, in get_latest_price_list
    current_price_list = binance_client.get_all_tickers()
  File "/local/AutoAltTrader/Trader/binance/client.py", line 381, in get_all_tickers
    return self._get('ticker/allPrices')
  File "/local/AutoAltTrader/Trader/binance/client.py", line 180, in _get
    return self._request_api('get', path, signed, version, **kwargs)
  File "/local/AutoAltTrader/Trader/binance/client.py", line 154, in _request_api
    return self._request(method, uri, signed, **kwargs)
  File "/local/AutoAltTrader/Trader/binance/client.py", line 148, in _request
    response = getattr(self.session, method)(uri, timeout=10, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 469, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 528, in urlopen
    **response_kw)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 284, in from_httplib
    **response_kw)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 79, in __init__
    self.headers = HTTPHeaderDict()
  File "/usr/lib/python3/dist-packages/urllib3/_collections.py", line 149, in __init__
    self.update(headers, **kwargs)
  File "/usr/lib/python3.4/_collections_abc.py", line 595, in update
    if isinstance(other, Mapping):
  File "/usr/lib/python3.4/abc.py", line 182, in __instancecheck__
    if subclass in cls._abc_cache:
  File "/usr/lib/python3.4/_weakrefset.py", line 72, in __contains__
    wr = ref(item)
RuntimeError: maximum recursion depth exceeded while calling a Python object

I am initializing the module in the normal way with;
import binance.client
binance_client = binance.client.Client(api_key, api_secret)

And calling binance_client.get_all_tickers() within an explicit "sched" loop;
def main():
    def run_sched_loop():
        s.enter(int(60), 1, run_sched_loop)  # Schedule next duty-cycle in 60 secs
        main_cycle()  # Do work (collect and store prices etc)
        try:
            s.run()  # Wait any remaining time, then call run_sched_loop() again
        except KeyboardInterrupt:
            logging.info('Manual break by user. Aborting')
            return 1
    run_sched_loop()

Within main_cycle() I am just calling "binance_client.get_all_tickers()" to build enough price history before I start doing anything fancy. However it fails before I can collect enough data.
I wonder if this is a result of using python-binance with the sched module? What do you think?
I am not closing the handle with session_close() each cycle or anything unusual, I'm just leaving it open.
Any suggestions are extremely welcome.
Kind regards, Andy.

I hope you to understand that I'm not good at English.
RuntimeError : maximum recursion depth exceeded while calling a Python object. < This error is caused by recursion. So I think its not python-binance that generating error is but sched module. Check and search for the error in Internet. Its maybe helpful.

The error is python based not the binance library. Your code is built using recursion and python has limits on recursion to prevent stack overflow. You can increase the recursion limit (not advised) using:
import sys
limit= ?
sys.setrecursionlimit(limit)

Per documentation:

Set the maximum depth of the Python interpreter stack to limit. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python.
The highest possible limit is platform-dependent. A user may need to set the limit higher when she has a program that requires deep recursion and a platform that supports a higher limit. This should be done with care, because a too-high limit can lead to a crash.

Id advise rewriting your function without recursion. If you're trying to create a loop that executes after a given amount of time you can use the time library and set a time.sleep() inside a while loop.
Additionally if its a matter of getting historical tickers just use the get_historical_klines function and set your specified time.


feat: init gitpod

closes #600
I'll probably need help with this:

[ ] The docker-compose needs to wait until the Stargate-Container runs before the Api can be started. This issue isnt Gitpod related, but I'm not deep enough into docker-compose to solve this straight.

Awesome! Thank you @Cahllagerfeld !!

The docker-compose needs to wait until the Stargate-Container runs before the Api can be started. This issue isnt Gitpod related, but I'm not deep enough into docker-compose to solve this straight.

No problem, we can ask for help on this, I am not sure either. Is a wait really needed though because I think the API does not make a connection to Stargate until a http request hits the API?

We should probably pair on this. I dont get it running with docker-compose :(

For me it worked when opening it in gitpod.
Its currently a bit dirty with the sleep, but for eddiecon it should work in my opinion

Awesome 👍 thank you @Cahllagerfeld 💥

What we should do is add the Gitpod-App to Eddiehub-Org so we can use the prebuild feature

What we should do is add the Gitpod-App to Eddiehub-Org so we can use the prebuild feature

We could ask Pauline to set this up properly, because its a quite powerful feature :D

I think you sent a request and I approved it to the org, if I need to do something else let me know

Should be done now, EddieHubCommunity Org has the GitPod app 🥳


Add new scenario into taurus load tests.

Initial scenarios was split in two:

login-and-open-intro-page (Edx LMS part)
work_with_problem (Adapt part)

Added new scenario:

xblock-enrolling (mix Edx LMS and Adapt requests)

Squash and merge to ensure it's going to be a single commit in master


Add check coinbase maturity for transparent outputs case

Now the original restriction to spend coinbases to transparent outputs is just disabled.
But we now should check coinbase maturity as well (for SomeTransparentOutputs case)

see main repo


Synchronise Email Inbox with Saltcorn

Hi,
first, thanks for the great tool!
I have a question: Is there a way to sync a Saltcorn app with an email inbox to process received emails. For example, if you receive an email, the Saltcorn app can fetch it through an event and then have it in a table?
Thank you very much,
Flo

Hi, this is certainly not something that is available now. See also #322. This will definitely happen at some point, because it's a great feature for CRM applications to be able to link client records to emails.
If anyone wants to contribute you could start by looking for libraries that could facilitate this. If we have a good library then it is likely easy to create an external table. we have an action to sync external and normal tables.

@glutamate  we can look to https://nodemailer.com/extras/mailparser/ . MIT license.
Creates JS objects from mail and give simple parser.
I can try to do implement this.
But question - is need to be part of Saltcorn core or this is will be external plugin?

@AleksandrSokolov this would be a plugin, because there are many different ways of getting mail - unix mailbox, IMAP, POP3, exchange etc

see https://github.com/saltcorn/system-info/blob/main/index.js#L48-L56 for an example of external table creation

Maybe external table is not the right thing, maybe we need an action that can contact a server and download unseen mail and copy to a real table. I am looking at https://github.com/mscdex/node-imap

I need this for my application! For this purpose I am developing a code in python that:

Analyzes the mailbox
Takes the following data: From, To, Subject, Date and Content
After that, it will send using the API (https://wiki.saltcorn.com/view/ShowPage/api) to send the data to the database
I started yesterday, the code is still very simple, but I can already get the data, now it's time to learn how to send the data to the API using python. hehehehe

This code in python access the IMAP server, read the mail in INBOX and print From, To, Subject and Body of the mail.
`import imaplib
import email
import email.parser
import email.policy
import html2text
import pprint
import sys
from datetime import datetime

change encode

def myprint(str):
print(str.encode('cp932', 'replace').decode('cp932'))

search mails in IMAP

def search_imap():
mail = imaplib.IMAP4_SSL("imap.gmail.com", 993)
<EMAIL_ADDRESS>"password")
mail.list()
mail.select('Inbox')  # specify inbox

# change ALL for UNSEEN for get only unseen mensages in INBOX
typ, [data] = mail.search(None, "(ALL)")

# for each mail searched
for num in data.split():
    # fetch whole message as RFC822 format
    result, d = mail.fetch(num, "(RFC822)")

    print("####################################")

    msg = email2Text(d[0][1])

    print(msg["from"])
    print(msg["to"])
    print(msg["subject"])
    print(msg["date"])
    # myprint(msg["body"])
    # Deleta o e-mail

mail.store(num, '+FLAGS', '\Deleted')
mail.expunge()
# closing
mail.close()
mail.logout()

Get subject, date, from and body as text from email RFC822 style string

def email2Text(rfc822mail):
# parse the message
msg_data = email.message_from_bytes(
    rfc822mail, policy=email.policy.default)

mail_value = {}

mail_value["from"] = header_decode(msg_data.get('From')).split("<")[
    1].split(">")[0]
mail_value["to"] = header_decode(msg_data.get('To'))
mail_value["date"] = header_decode(msg_data.get('Date'))
mail_value["subject"] = header_decode(msg_data.get('Subject'))
mail_value["body"] = ""
if msg_data.is_multipart():
    for part in msg_data.walk():
        ddd = msg2bodyText(part)
        if ddd is not None:
            mail_value["body"] = mail_value["body"] + ddd
else:
    ddd = msg2bodyText(msg_data)
    mail_value["body"] = ddd

return mail_value

get body text from a message (EmailMessage instance)

def msg2bodyText(msg):
ct = msg.get_content_type()
cc = msg.get_content_charset()
cte = msg.get("Content-Transfer-Encoding")
# skip non-text part/msg
if msg.get_content_maintype() != "text":
    return None

ddd = msg.get_content()

# html to text
if msg.get_content_subtype() == "html":
    try:
        ddd = html2text.html2text(ddd)
    except:
        print("error in html2text")

return ddd

get the real information on the e-mail variables

def header_decode(header):
hdr = ""
for text, encoding in email.header.decode_header(header):
if isinstance(text, bytes):
text = text.decode(encoding or "us-ascii")
hdr += text
return hdr
if name == "main":
search_imap()
`

@Tacioandrade I've published client library for python: https://pypi.org/project/pysaltcorn/
All saltcorn API entrypoints implemented, but not documentation yet.


Configurator - Add Settings Page

Add page to the config app to change runtime settings like D-pad mode, SOCD mode, etc.

Page has been added.


Catering

decide on catering, depends on number of sponsors and available budget

@erikauer could you plase add the details from your contacts

@erikauer: I think we can close this issue right?
Are the documents already in the Google Drive?


Integrated new Signup Flow

Issue
When signing in with GitHub send the page href in query params as &state=

RFC - https://github.com/Real-Dev-Squad/website-backend/issues/688

Description
When you visit the main site and click on login, you are redirected to the original GitHub login page. We want to emulate a similar scenario for the status site.
Expected Flow

User visits welcome-site for the first time
User clicks on sign-in-with-github
Now, they are redirected to the GitHub login page
once the login is successful the user is expected to land back to the status site
Example PR - https://github.com/Real-Dev-Squad/website-www/pull/277

It is already fixed here https://github.com/Real-Dev-Squad/website-status/issues/349 so, we can close this issue


Docs: Learn Next.js Chapter 9

What is the documentation issue?
Chapter9

According to the context , we didn't add any skeletons to the customers pages ,so customers pages won't display skeleton.
Is there any context that might help us understand?
Please delete and customers pages as well. As the guide did ,only invoices page will loading skeleton.
Does the docs page already exist? Please link to it.
https://nextjs.org/learn/dashboard-app/streaming

export default async function Page (){
await new Promise((resolve) => setTimeout(resolve, 3000));
return (
Invoices Page
)
}
I hope it will be help you.
If you want to cooperate with me, contact me.


System.Decimal string.format issue

From @ironpythonbot on December 9, 2014 18:12
When I run the following program, I cannot format a System.Decimal. I have an
integration with a .NET library that returns a decimal type that I need to
display in the ironpython script.
import System
f = 1.42742
d = System.Decimal(f)
print "{}".format(d)            # 1.42742
print "{:,.2f}".format(f)       # 1.43
print "{:,.2f}".format(d * 1.0) # 1.43
print "{:,.2f}".format(d)       # 12f WRONG

Work Item Details
Original CodePlex Issue: Issue 34710
Status: Proposed
Reason Closed:  Unassigned
Assigned to:    Unassigned
Reported on:    Nov 30, 2013 at 1:34 PM
Reported by:    davcar
Updated on: Dec 4, 2013 at 4:08 PM
Updated by: jdhardy
Copied from original issue: IronLanguages/main#1093

From @ironpythonbot on December 9, 2014 18:12
On 2013-12-03 01:51:36 UTC, jdhardy commented:
Can you try converting to a decimal.Decimal first?
d = decimal.Decimal(System.Decimal(f))
Handling System.Decimal directly is preferable but this might at least give you a workaround.

From @ironpythonbot on December 9, 2014 18:12
On 2013-12-03 02:44:22 UTC, davcar commented:
Converting it as you suggested works. However I am actually receiving a lot of .Net decimal types from a .Net method I am calling. I was hoping not to have to do any extra conversions for each. Currently to avoid that, I am doing the conversion to a double
inside the .Net method which works but not really great for using as money.

From @ironpythonbot on December 9, 2014 18:12
On 2013-12-03 18:07:48 UTC, vernondcole commented:
Given that 'string'.format() is a function from the Python standard library, I would say that the chance of getting the worldwide Python community to embrace the extension you suggest for a foreign (dotNet) data type is practically nil -- especially since
the option, converting to decimal.Decimal is both easy, and specially supported by the IronPython runtime for exactly this kind of situation. Conversion to Double could loose precision, direct conversion will not.
from System import Decimal as SystemDecimal
import decimal
sd = SystemDecimal(1234567)
thousand = SystemDecimal(1000)
fract = sd / thousand
dd = decimal.Decimal(fract)
print('{:,.2f}'.format(dd))
1,234.57
Works just exactly like you want.

From @ironpythonbot on December 9, 2014 18:12
On 2013-12-03 02:46:10 UTC, davcar commented:
By the way, I am using IronPython as a way to let the user store scripts in the database to do some simple summary reporting with.

From @ironpythonbot on December 9, 2014 18:12
On 2013-12-05 00:08:28 UTC, jdhardy commented:
The format protocol allows types to have a
format method that lets them participate without chaning the built-in format function. There's an extension mechanism, also used for System.Double and System.Int32, that allows adding those special methods to .NET types so that they can
interact with Python facilities.
We wouldn't need to get any other Python implementation to support System.Decimal; it's just a matter of adding the appropriate extension in IronPython. I'd have to dig into the code to remember exactly how to do it, but I know it's possible.


Client did not load private key

I used the KeyStore Explorer to add an RSA key and a "CN=CNG 2 Client" Certificate to the Windows-MY store:
The client finds the certificate but not the related private key:
C:\git\cng-openssl-provider\custom-build-directory\client\Debug>client.exe
PROGRAM> We will connect to a remote server and check the SSL certificate
Configuration in section openssl_init
Adding config module 'alg_section'
Adding config module 'providers'
Adding config module 'random'
Loading providers module: section provider_sect
Configuring provider default
Provider params: start section default_sect
Provider params: finish section default_sect
Running module providers (provider_sect) returned 1
Looking up scheme cng
Found loader for scheme cng
Opened cng://MY => 0x24a5a2da3c0
Loading next object
Got a Certificate
PROGRAM> Found certificate in store
Loading next object
Got a Certificate
PROGRAM> Found certificate in store
Loading next object
Got a Certificate
PROGRAM> Found certificate in store
Loading next object
Got a Certificate
PROGRAM> Found certificate in store
Loading next object
Got a Certificate
PROGRAM> Found certificate in store
Loading next object
Got a Certificate
PROGRAM> Found certificate in store
Closing 0x24a5a2da3c0
Looking up scheme cng
Found loader for scheme cng
Opened cng://MY => 0x24a5bfe5710
Loading next object
Got a Certificate
Loading next object
Got a Certificate
Loading next object
Got a Certificate
Loading next object
Got a Certificate
Loading next object
Got a Certificate
Loading next object
Got a Certificate
Loading next object
Closing 0x24a5bfe5710
PROGRAM> Could not find matching private key in store
PROGRAM> Client exiting...
Cleaned up providers

My OpenSSL version is 3.2.0. Whats did I wrong?
Thanks in advance!
@ddvo

I'm noticing that the key in question is locked...  not sure how this works, but could that be a problem?

Unfortunately I was not able to reproduce your problem. Maybe it has to do with the KeyStore explorer. Please try to import the certificate into the CNG store with the normal Windows tools. A pkcs12 certificate with an associated key can be easily imported by just double clicking on in it in Windows explorer. Then you can even specify whether or not should the user be prompted for a password on each key usage.
Then when using the provider you should see the line Got a Pkey. The fact that this line is missing in your log suggests that maybe the associated private key did not get imported into the CNG store.
Try looking into the certmgr utility on windows, navigate to Personal certificates and tell me if you can see the line You have a private key that corresponds to this certificate.. It appears at the bottom of the text input and looks like this:

Sorry for late response. Las week I was sick.

Thanks for the debug output. I get:
C:\git\cng-openssl-provider\custom-build-directory\client\Debug>client
PROGRAM> We will connect to a remote server and check the SSL certificate
Configuration in section openssl_init
Adding config module 'alg_section'
Adding config module 'providers'
Adding config module 'random'
Loading providers module: section provider_sect
Configuring provider default
Provider params: start section default_sect
Provider params: finish section default_sect
Running module providers (provider_sect) returned 1
Looking up scheme cng
cng_store_open
STORE> The system store is open. Continue.
Found loader for scheme cng
Opened cng://MY => 0x22ebde5bb10
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
cng_store_eof
Closing 0x22ebde5bb10
cng_store_close
PROGRAM> Could not find certificate with this common name in store
PROGRAM> Client exiting...
Cleaned up providers
Cleaned up providers

So the client finds the certificates on my company smart card (there are also private keys) but not the certificates shown by the certmgr at  "Certificate - Local Computer -> Personal -> Certificates".
If I use
#define SEARCH_FACTOR NID_serialNumber
#define SEARCH_VALUE "ZZZZZGXC"

I get
C:\git\cng-openssl-provider\custom-build-directory\client\Debug>client.exe
PROGRAM> We will connect to a remote server and check the SSL certificate
Configuration in section openssl_init
Adding config module 'alg_section'
Adding config module 'providers'
Adding config module 'random'
Loading providers module: section provider_sect
Configuring provider default
Provider params: start section default_sect
Provider params: finish section default_sect
Running module providers (provider_sect) returned 1
Looking up scheme cng
cng_store_open
STORE> The system store is open. Continue.
Found loader for scheme cng
Opened cng://MY => 0x1b6b6ffb4e0
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
Closing 0x1b6b6ffb4e0
cng_store_close
Looking up scheme cng
cng_store_open
STORE> The system store is open. Continue.
Found loader for scheme cng
Opened cng://MY => 0x1b6b8d859f0
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
cng_store_eof
Closing 0x1b6b8d859f0
cng_store_close
PROGRAM> Could not find matching private key in store
PROGRAM> Client exiting...
Cleaned up providers
Cleaned up providers

The client finds the certificate but not the key.
So I would summarize:

The client looks for certificates into the Windows-MY store
The "Crypto Shell Extensions" (doubleclick) on my computer is not able to import certificates into the Windows-MY store, it saves it somewhere else
The certmgr shows certificates imported by the "Crypto Shell Extensions" but not the content of the Windows-MY
The KeystoreExplorer is able to import certificates and keys into the Windows-MY store
If the client finds a matching certificate it is not able to load any private key

Maybe the behavior depends an the Windows version or some company policies.

@DDvO does the thumbs up mean that this solved your problem and I can close the issue, or that you have interest in other stores than the user store?

My 'thumbs up' in this case just means that I'm glad a solution apparently was found.
I did not try myself, but I'm sure @Akretsch will do and confirm if it works now for him.

Now I used the user certificate manager to import a selfsigned CN = CNG 2 Client certifcate and key. Is visible beside my smart cart certificates at Certificates - Current User -> Personal -> Certificates.   is also there.
Unfortunately the situation did not improve:
**C:\git\cng-openssl-provider\custom-build-directory\client\Debug>client.exe
PROGRAM> We will connect to a remote server and check the SSL certificate
Configuration in section openssl_init
Adding config module 'alg_section'
Adding config module 'providers'
Adding config module 'random'
Loading providers module: section provider_sect
Configuring provider default
Provider params: start section default_sect
Provider params: finish section default_sect
Running module providers (provider_sect) returned 1
Looking up scheme cng
cng_store_open
STORE> The system store is open. Continue.
Found loader for scheme cng
Opened cng://MY => 0x23093402f60
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
PROGRAM> Found certificate in store              <==================================================
CN=CNG 2 Client
Closing 0x23093402f60
cng_store_close
Looking up scheme cng
cng_store_open
STORE> The system store is open. Continue.
Found loader for scheme cng
Opened cng://MY => 0x230951c7290
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
Got a Certificate
cng_store_eof
cng_store_eof
Loading next object
cng_store_load
cng_store_eof
Closing 0x230951c7290
cng_store_close
PROGRAM> Could not find matching private key in store  <===================================
PROGRAM> Client exiting...
Cleaned up providers
Cleaned up providers

Yeah, that is unfortunate.
I've tried to change the debug output to something more useful. Please try to compile the new version of the client and run it again. Hopefully it will give us more insight.

C:\Users\kretscha>cd \git\cng-openssl-provider\custom-build-directory\client\Debug

C:\git\cng-openssl-provider\custom-build-directory\client\Debug>client
PROGRAM> We will connect to a remote server and check the SSL certificate
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
CN=CNG 2 Client
PROGRAM> The certificate matches search factor. Trying to load it into SSL context.
PROGRAM> Public key of the certificate is:  Public-Key: (2048 bit)
 Modulus:
     00:d0:cf:d9:2e:6e:9d:0e:12:43:d7:a9:33:1f:0e:
     b7:b4:61:e0:26:c4:42:e0:e6:41:be:02:9e:20:58:
     e2:95:0e:32:bf:3f:f0:a4:2e:3c:d3:2a:51:11:35:
     87:04:49:ae:4b:36:55:50:81:bb:65:0d:ac:05:fe:
     5b:7e:37:09:e9:42:f8:e5:6d:bd:8a:74:85:c8:8b:
     37:06:bd:1b:66:e3:7f:86:76:36:33:eb:3e:b4:2d:
     1e:4e:77:55:c0:10:e2:4b:92:bb:f7:64:63:a8:08:
     f2:fc:d7:8b:0e:ca:bb:6f:dd:fc:dd:19:36:af:da:
     b7:da:32:44:17:f4:f9:e1:66:ca:b1:b0:ef:3c:d3:
     32:89:fe:21:20:45:fb:ac:23:9e:6d:e6:02:14:85:
     59:af:64:36:a6:d0:6a:b3:83:59:5c:4e:2b:b8:aa:
     43:0f:11:8e:6a:b4:7f:b4:7e:73:e7:e8:15:82:80:
     2e:54:3e:30:90:27:2d:b4:17:53:db:d3:bc:68:a0:
     93:ae:53:cd:76:e4:a8:cc:35:91:73:5c:1b:69:1d:
     e6:68:f0:25:cf:dd:52:75:8b:ef:c4:e3:79:28:ac:
     5e:7a:2c:a6:4b:5b:42:3e:8e:07:4c:d7:ce:7e:2a:
     8d:86:7a:36:b8:82:87:ad:8f:b6:61:96:f2:ff:48:
     66:c3
 Exponent: 65537 (0x10001)
PROGRAM> Certificate successfully loaded into SSL context
PROGRAM> Could not find matching private key in store
PROGRAM> Client exiting...

Unfortunly the next 2 weeks I will be away without access to my Laptop.

I have another idea! Do you perhaps have a certificate that has an ECC not an RSA key associated with it in your CNG store?

As can be seen from the above output priovded by @Akretsch, he was using a 2048-bit RSA key.

I have another idea! Do you perhaps have a certificate that has an ECC not an RSA key associated with it in your CNG store?

As can be seen from the above output provided by @Akretsch, he was using a 2048-bit RSA key.

Yes, I can see that. But if the store contains an ECC key as well as an RSA key it might break things.

Now I understand what you meant.
If the store contains both RSA and EC keys/certs but with different identifiers this should not be a problem.
Anyway he only talked abut RSA keys, and his other (company) key material is RSA-only.
So I'm pretty sure the issue is something different.
Since the Windows KeyStore explorer is able to relate to the test cert a corresponding private key, your tool should be as well.

I have not tested the provider with password protected keys. I'll try to generate myself a few of these and get back to you.

Did you meanwhile test positively with a key that is password-protected?

I have not tested the provider with password protected keys. I'll try to generate myself a few of these and get back to you.

Did you meanwhile test positively with a key that is password-protected?

Yes, keys that need password input on each use work with this provider. You will see the usual Windows dialog box pop up when you use them.

Now I understand what you meant. If the store contains both RSA and EC keys/certs but with different identifiers this should not be a problem. Anyway he only talked about RSA keys, and his other (company) key material is RSA-only. And he certainly did not place RSA and EC keys/certs with the same identifier.
So I'm pretty sure the issue is not about RSA vs. EC keys/certs. Since the Windows KeyStore explorer is able to relate to the test cert a corresponding private key, your tool should be as well.

When an ECC key is in the store it will be a problem for the provider during enumeration.
But as this is not the case and I’m unable to recreate it. Would you mind sending me the certificate you have generated so I can try it myself? @Akretsch
Thanks

@Lipovlan, it appears that your code assumes that all keys are RSA keys.  That's quite clear by just looking at this line, that declares every key you load as an RSA key, unchecked:
https://github.com/Lipovlan/cng-openssl-provider/blob/398dedd79d43e8364671a0f09c62f8c545bcec8c/cng_provider/store/cng_store_functions.c#L247
It would be good if this function checked that the CNG key type indicates that it's an RSA key and only "load" it if it is.

Yeah good idea! I've implemented the check. @Akretsch should be back soon, so I'm wondering if this solves his problem.

So the store, in this case, is a PKCS#12 file.  Isn't this kinda sorta hinted at in #5? (CertOpenStore() can open a PKCS#12 file, while CertOpenSystemStore(), which is currently used, cannot if I read the docs correctly.

As already written in https://github.com/Lipovlan/cng-openssl-provider/issues/4#issuecomment-1562371968 :
I used the user certificate manager to import the selfsigned CN = CNG 2 Client certifcate and key from the .p12.

The result is still:
C:\git\cng-openssl-provider\custom-build-directory\client\Debug>client.exe
PROGRAM> We will connect to a remote server and check the SSL certificate
STORE> The system store is now open.
STORE> Trying to preload certificates from store.
STORE> Trying to preload private keys from store.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
serialNumber=ZZZZZGXC, GN=Andreas, SN=Kretschmer, O=Siemens, CN=Kretschmer Andreas
PROGRAM> The certificate does not match search factor. Skipping.
PROGRAM> Found certificate in store
CN=CNG 2 Client
PROGRAM> The certificate matches search factor. Trying to load it into SSL context.
PROGRAM> Public key of the certificate is:  Public-Key: (2048 bit)
 Modulus:
     00:d0:cf:d9:2e:6e:9d:0e:12:43:d7:a9:33:1f:0e:
     b7:b4:61:e0:26:c4:42:e0:e6:41:be:02:9e:20:58:
     e2:95:0e:32:bf:3f:f0:a4:2e:3c:d3:2a:51:11:35:
     87:04:49:ae:4b:36:55:50:81:bb:65:0d:ac:05:fe:
     5b:7e:37:09:e9:42:f8:e5:6d:bd:8a:74:85:c8:8b:
     37:06:bd:1b:66:e3:7f:86:76:36:33:eb:3e:b4:2d:
     1e:4e:77:55:c0:10:e2:4b:92:bb:f7:64:63:a8:08:
     f2:fc:d7:8b:0e:ca:bb:6f:dd:fc:dd:19:36:af:da:
     b7:da:32:44:17:f4:f9:e1:66:ca:b1:b0:ef:3c:d3:
     32:89:fe:21:20:45:fb:ac:23:9e:6d:e6:02:14:85:
     59:af:64:36:a6:d0:6a:b3:83:59:5c:4e:2b:b8:aa:
     43:0f:11:8e:6a:b4:7f:b4:7e:73:e7:e8:15:82:80:
     2e:54:3e:30:90:27:2d:b4:17:53:db:d3:bc:68:a0:
     93:ae:53:cd:76:e4:a8:cc:35:91:73:5c:1b:69:1d:
     e6:68:f0:25:cf:dd:52:75:8b:ef:c4:e3:79:28:ac:
     5e:7a:2c:a6:4b:5b:42:3e:8e:07:4c:d7:ce:7e:2a:
     8d:86:7a:36:b8:82:87:ad:8f:b6:61:96:f2:ff:48:
     66:c3
 Exponent: 65537 (0x10001)
PROGRAM> Certificate successfully loaded into SSL context
STORE> The system store is now open.
STORE> Trying to preload certificates from store.
STORE> Trying to preload private keys from store.
PROGRAM> Could not find matching private key in store
PROGRAM> Client exiting...

CMS....  is a more difficult beast.  It lacks full provider support in some areas, which is something I'm going to talk about next week.

@Akretsch I have tried to use your key and unfortunately the code still works for me. I will try to spin up a VM and try a clean setup. Hopefully I'll be able to get some results.
And I'm really glad, that you considered this provider as a stepping stone! That is what I had in mind when creating it :)

I just had a closer look at the error output.
According to The certificate does not match search factor. Skipping.
and your code, you only accept certs that have in their subject the common name "CNG 2 Client",
so no wonder it works (just) for you :wink:

Ah, Andreas pointed out to me what he wrote before that he also tried using your test certs, with correctly matching common name, but even this failed for him.
Anyway, do not hard-code a restriction on the common name of the certificates loadable by your provider.
This selection should be done otherwise, e.g. by "cng:file-path-name"

Indeed. Andreas has even generated his own certificates and changed the relevant parts of declarations in the code as far as I'm aware.
The certificate name should be part of the scheme as you pointed out, yet there wasn't time to implement that. I'm thinking about something along the lines of adding to the current scheme cng://store-type identifier type-value pair like cng://store-type/identifier-type/identifier-value. So addressing via cng://MY/commonName/CNG 2 Client should be possible.

I have no idea what CMS means in this context but hooray!
Also I realized, that the provider itself does not have the common name hard-coded. Only stuff in the "test application" that is a TLS client has stuff hard-coded and I won't be changing that any time soon.
The provider just enumerates what is in one of the select-able CNG stores, so that should not pose a problem.

I have no idea what CMS means in this context but hooray!

@Akretsch was considering using your provider implementation as a model for his provider
to help fixing issues like these: https://github.com/openssl/openssl/issues/20845
where using his provider did not work for CMS signing.

Also I realized, that the provider itself does not have the common name hard-coded. Only stuff in the "test application" that is a TLS client has stuff hard-coded and I won't be changing that any time soon.

Good point.
Yet Andreas was using your test client, so among others he faced this IMHO needless issue.

The provider just enumerates what is in one of the select-able CNG stores, so that should not pose a problem.

I see - for general use of the provider, this ist fine.


NETStandard as global dependency

From @mrapan on Monday, December 19, 2016 7:15:20 AM
It looks like NETStandard.Library is a global dependency in project.json, this leads to .NET Core dependencies even if targeting full framework.
Copied from original issue: aspnet/Proxy#36

From @cwe1ss on Monday, December 19, 2016 7:25:22 AM
See https://docs.microsoft.com/en-us/dotnet/articles/standard/library
NETStandard.Library is a meta-package. The full framework will use its own libraries.


More than one option flagged as selected in single-select matcher

Steps to reproduce the issue

Create a single-select with a 4 options
Add custom matcher that filters out selected items (data.selected === true)
Select an option, then another option

Expected behavior and actual behavior
I would expect to see a selected option in the picker box, and 3 more options in the dropdown. What I actually see is ALL previously selected items are removed from the dropdown... Making more selections just reduces the available options until there are none left.
Environment
Browsers

[x] Google Chrome
[x] Mozilla Firefox
[ ] Internet Explorer (not tested)

Operating System

[x] Mac OS X
others not tested

Libraries

jQuery version: ?
Select2 version: 4.0.3

Without seeing your code for the matcher, preferably in a jsbin so we can test it live, there isn't much we can do. You've narrowed it down to an issue when using a custom matcher, but that's about all we have to go on right now.

@kevin-brown Here is a codepen showing the issue... If I'm doing something wrong in the matcher, let me know, but it seemed to match the examples: http://codepen.io/jessehouchins/pen/WxkjNa

FYI, I see the same behavior (more than one option set to selected) when stepping through the built in matcher in v4.0.3.

So, the data.selected property does get out of sync with the actual selection, but I can just as easily use data.element.selected for my purposes. If there cases where that is not an option though, it might be worth fixing.


Soap Headers and Empty Calls

Added soap headers and added a fix for empty calls without parameters.

Thanks, @earncef for the contribution!


page 1500 Workflows

Hi,
Can you please make these small following changes in page 1500 Workflows, we need to access these variables and procedures in page extension.
Before
    var
        ExportEnabled: Boolean;

After
    protected var
        ExportEnabled: Boolean;

Before
local procedure RefreshTempWorkflowBuffer()

After
procedure RefreshTempWorkflowBuffer()

Before
local procedure GetFilterFromSelection() "Filter": Text

After
procedure GetFilterFromSelection() "Filter": Text

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 46680.


scene initial zoom

In v9 I was able to save the initial view position with a zoom less than 0.01, but in v10 (with  maximum/minimum zoom override set to 150) it keeps bumping up t0 0.25 as a minimum.

I investigated a little, found that the core foundry scene has a minimum initial-view-position zoom level of 0.25 which I don't know how to change.  So, this will be a known bug for now, no plan to fix unless someone else has an idea.


Resetting audio sliders

It seems using a graphics config resets the audio sliders without an option to customize them.
Every time I launch an alt through LB my sliders are all messed up.

Launchbuddy doesn't store or change the audio settings, so they're probably stored in "%appdata%\Guild Wars 2\local.dat". Which Launchbuddy doesn't touch.

After some testing it seems I was mistaken.
Changes to audio sliders done through LaunchBuddy are only stored for the current instance.
To do longer lasting changes I have to open up the client normally and edit the sliders there.


Rooftop antennas in home world aren't solid

This guy on the left:

Leads to some fun:

Haven't checked the others because they're taller and more difficult to get to

confirmed the taller antenna also isn't solid


On iOS switching to RN60.0 causes error: React/RCTDefines.h file not found

Switching to RN60.0 the library is not working. I added to Libraries the TfliteReactNative.xcodeproj file and added libTfliteReactNative.a to Build phases. After the build fails, with this issue.
The changes in the switch from RN59.10 to RN60.0 version can be seen here:
https://react-native-community.github.io/upgrade-helper/?from=0.59.10&to=0.60.0
For anyone helping solve the issue:

/react-native/React/Base/RCTBridgeModule.h:10:9: 'React/RCTDefines.h' file not found

{PROJECT_PATH}/node_modules/tflite-react-native/ios/TfliteReactNative.mm:2:9: In file included from {PROJECT_PATH}/node_modules/tflite-react-native/ios/TfliteReactNative.mm:2:
{PROJECT_PATH}/node_modules/tflite-react-native/ios/TfliteReactNative.h:3:9: In file included from {PROJECT_PATH}/node_modules/tflite-react-native/ios/TfliteReactNative.h:3:

On Android things work just as they should.
My guess on the issue:
When we switch to RN60.0 podfile, we lost some of it's includes, and tflite-react-native library on some of these React-Core pods files. But still I don't know what is missing and how to correct it.

I opened a pull request to update the lib for react-native 0.60. If you have time to try it, give me some feedback !

Any updates on this issue?

#1 Try to change the way of import:
#if __has_include("RCTDefines.h")
#import "RCTDefines.h"
#else
#import <React/RCTDefines.h>
#endif


[Global] Move author div computation to "something server side"

Upcoming change https://github.com/davidnuescheler/theblog/blob/alexonedrive/head.html#L32-L65 deals with the "by AUTHOR" text in the DOM, fetch the author avatar and construct the expected DOM for an author. This is a decent amount of logic which is run client side. Which means this final DOM will not be cached by Fastly and we be recomputed on each request.
This could be moved into a pre.js but we are using helix-pages here which does not support it. https://github.com/adobe/helix-pipeline/issues/516 will introduce one step into this direction but not enough to cover this use case (see my comment there).

The author request when the file is stored in onedrive is so slow that you see the author appearing later on... see https://author--theblog--davidnuescheler.hlx.page/posts/adobe-named-a-leader-in-forresters-latest-enterprise-marketing-software-suites-report.html

@kptdobe why do you need an extra (onedrive) request for this? every thing you need to know should be available either from the markdown or the query.

everything you need to know should be available either from the markdown

That's the problem, it is not. The blog post markdown contains only the name of the author. Via Helix-Pages, only way to resolve the author's avatar is, base on this name, compute the author page path, make a new request from the client to get the author page and inject the image into the current dom.
And this is true for any resolution that needs to happen (like icons of product list...). We already have multiple of them, then multiple extra requests to onedrive.
In production, each of these requests will be cached but the blog post final dom will not be, it will have to be recomposed for each surfer accessing every single blog post entry...

I would imagine,
by Major Domo

--> src="/autors/major_domo/avatar.png"

(like icons of product list...)

why? the icons are not stored in one-drive, they are given by the designer.

Yes and no. This could be one approach but requires a lot of discipline for the authors to create their bio (must be a png, must be named avatar...).
For the author and the products, we need a bit more than the image:

author biography
product link to the adobe.com site

Thus to store this content, we need to create page per author and per product which then I think should contain all content, including the image / logo, a potential display name (including accents, job title...), bio / product description. This also allows to have an entry point from requests /authors/major_domo.html which has to be served anyway.
From there, 1. it is not only about the image only 2. we cannot compute the image path automatically because it is stored in the docx file.
Another thing is that for now, you cannot store anything else than .docx (and soon .md) in OneDrive / Google Doc. Having images in Git and bio in OneDrive does not make sense.

Another thing is that for now, you cannot store anything else than .docx (and soon .md) in OneDrive / Google Doc. Having images in Git and bio in OneDrive does not make sense.

the author images are uploaded to azure anyways during processing. and those remain stable. so the  index can store the image urls as a property for the author page.

Our discussion today led to the following conclusion: all content collection and mapping should happen via the index (see #23): we'll then have only one extra request to Algolia and the client side code will only feed the dom with the content provided from here.


Fix for Chef 12.1.0 masking exceptions in converge.

This is being tracked upstream in https://github.com/chef/chef/issues/3078. The behavior of converge changed from re-raising exceptions to returning them. This will hopefully be resolved in 12.1.2 but this code will be needed to match things up for 12.1.0 and 12.1.1. Without this, you cannot test for exceptions being raised by recipe code.

:+1: reboot is not present in 11.0 , iirc. @sethvargo should we tag these specs 12.x only ?

The relevant test that make me notice this is https://github.com/poise/poise/blob/master/test/spec/notifying_block_spec.rb#L68-L86

@coderanger thanks!
@ranjib yea, do you want to merge this and fix it up on master? If not I can do it later. We should definitely tag those specs :smile:

@sethvargo i can, but you have to wait till next weekend.

That's fine :). You'll probably beat me to it

:+1:

https://github.com/sethvargo/chefspec/commit/262c7b597da202a1540c79d4805c71fc97536834


bug ID

Oops! An error occured.
[ DEBUG MODE ]
MESSAGE:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"0", 25' at line 1
SELECT id, username FROM sw_users ORDER BY username LIMIT "0", 25;
FILE: /var/www/sw/_app/plugins/Db.php
LINE: 346
NO: 1064
CWD: /var/www/sw
[ BACKTRACE ]

Sorry I ignored your report until now, I just started working on v3 from scratch so this no longer applies.


lodctr parameter /m: is undocumented

Excerpt from lodctr /?:
    LODCTR /M:<Counter-Manifest>
        install Windows Vista performance counter provider definition XML file
            to system repository.

Note that the above description is incomplete, as well: when lodctr is called with /m:, it also takes an optional parameter to specify the path of the directory that contains the resource DLL with the performance counter names and descriptions. Yes, just the directory: the file name is taken from the manifest, and it's the contents of the applicationIdentity attribute of the <provider> element. If this optional argument isn't passed correctly, the performance counters so registered may be unusable
I have no idea what team owns lodctr and if they have a publicly accessible issue tracker, but they should know about the incorrect/incomplete built-in help in the tool

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d4b3ac89-b24b-e0c9-3c32-6f341e99d256
Version Independent ID: 5fbaecde-5418-5179-9690-17f87843a131
Content: lodctr
Content Source: WindowsServerDocs/administration/windows-commands/lodctr.md
Product: windows-server
Technology: manage-windows-commands
GitHub Login: @coreyp-at-msft
Microsoft Alias: coreyp

We hugely appreciate your contribution, feedback, and support to our technical documentation.
We’ve phased out GitHub Issues on this repository as the feedback mechanism for content and have replaced it with a new feedback system. For more information see, Content User Feedback. As part of this rollout, we are migrating your issue where our team will use your comments to improve the documentation experience. Unfortunately, this does mean you won't be able to access this issue after the migration is complete.
All issues will be migrated by Thursday 29th August 2024.
Once again, thank you for your valuable feedback, we hope you’ll continue to contribute using our new feedback system.


Add example for user specified scales

Create an example which demonstrates how users can scale axes (especially, in conjunction with keep/remove data methods). This would use the pc.scale() method from Parcoords which will eventually be made available to Parasol (i.e., ps.scale()).

I've implemented the pc.scale() method to scale the axes in the LRGV example for the paper. So this task is complete.


Highcharts 6.2.0 breaks Series componentWillUnmount

Bug or Feature Request?
Bug. Might be in highcharts itself.
Description
Upgrading to Highcharts 6.2.0 breaks Series componentWillUnmount.
https://github.com/whawker/react-jsx-highcharts/blob/9c20dfbc12e2f16a12e5c1fe4ff1ae3da61b8ae0/packages/react-jsx-highcharts/src/components/Series/Series.js#L78
Error:
Cannot read property 'bind' of null

How to reproduce
Any series unmount triggers this.
Live demo demonstrating bug
Navigate to series and back to index on:
https://stackblitz.com/edit/github-vtzme3?file=src/index.js
Versions

React JSX Highcharts version: 3.2.1
Highcharts version: 6.2.0
React version: 16.5.2

Probably this commit in highcharts:
https://github.com/highcharts/highcharts/commit/dc77c44c1a87109ef7e8580e4b55be1989d075df
The series.remove is now set to null after series.destroy()

Legend, thanks @anajavi!

This is very likely to happen in future in Axis:
https://github.com/whawker/react-jsx-highcharts/blob/4649e05a1aaa7d31c3deab654c5f72a30d2e1b63/packages/react-jsx-highcharts/src/components/Axis/Axis.js#L67
Do you think it should be wrapped too?

Yeah, it's probably best we guard against it.

Thanks @anajavi, I'll publish 3.3.0-beta.1 now.

Quick testing with chrome & IE11 seems to be okay.

Find out something that helps debugging. Importing Highcharts from es-module version gives file and line number when it throws:
import Highcharts from 'highcharts/es-modules/masters/highcharts.src';
import 'highcharts/es-modules/masters/highcharts-more.src';

It also gives slightly smaller app bundle.


add Open ❤  Source Badge

Thanks. Honestly this seems a little redundant, as the entirety of GH is :heart: open source.


[enhance](buffered reader) use stop token to simplify state change check code logic

Proposed changes
Issue Number: close #xxx
The states of the buffered reader are as follows:

But formerly we had no elegant way to check if the state has transferred to close, now C++ 20 supports stop resource and stop token, we can simply integrate them with condition variable any to simplify the code.

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 42.81 seconds
stream load tsv:          463 seconds loaded<PHONE_NUMBER>9 Bytes, about 154 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          58 seconds loaded<PHONE_NUMBER> Bytes, about 18 MB/s
stream load parquet:          29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select:          68.3 seconds inserted 10000000 Rows, about 146K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230703114153_clickbench_pr_171472.html

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.38 seconds
stream load tsv:          516 seconds loaded<PHONE_NUMBER>9 Bytes, about 138 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select:          29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size:<PHONE_NUMBER>7 Bytes


Failed to set to the execution context

Hi axodox,
I want to run the test, but all of the test results show "Failed to set to the execution context."
I have run update_dependencies.ps1 and build_nuget.ps1.
Did I miss something?
Thanks!

That is because Windows ships an old version of DirectML and ONNX runtime, and it is loaded instead the latest Nuget into the test runner. A workaround is to copy these files in the folder of the test runner.
However you will also need other input files and models to run the tests, which are currently not included in the repo due to their size, and GitHub LFS costs.


Wrong topic URL when Gitea installed in the subdomain

Gitea version (or commit ref): 513db27 and older
Database:

[x] SQLite

Can you reproduce the bug at https://try.gitea.io:

[x] No

Description
When Gitea is installed in the subdomain, "Topics" visible on the home page have the wrong URL to route to the main domain. This error can not be repeated in the trial version of TryGitea because it is not in the subdomain.

-> https://github.com/go-gitea/gitea/pull/5021


App crashes when activating Push Notification, after upgrading to v5 iOS

Bug Report
App crashes when activating Push Notification, after upgrading to v5
Expected Behaviour
activating push notification, open the app, and activates correct events
Actual Behaviour
app crashes when opened from a push notification
Reproduce Scenario (including but not limited to)
activate push notification from iOS
Steps to Reproduce
Platform and Version (eg. Android 5.0 or iOS 9.2.1)
iOS 18.1 latest version of the plugin
(Android) Device Vendor (e.g. Samsung, HTC, Sony...)
iPhone 14 Pro
cordova info Printout
Im not confident in providing the cordova info, as I can see it contains private information to our production app
Sample Push Data Payload
Payload is structered 1:1 as in the documentation for the plugin, we have been using the plugin for many many years
Sample Code that illustrates the problem
Logs taken while reproducing problem
The error happens in this line
https://github.com/havesource/cordova-plugin-push/blame/12a5b71cb381c8e0ad0370a771ac523dc46de7a3/src/ios/PushPlugin.m#L497
-[__NSDictionaryI removeObjectForKey:]: unrecognized selector sent to instance 0x301b8db80

I am currently unable to reproduce this issue.
The code you're referring to is triggered when a notification is received, but you're implying that the issue occurs when the push notification is activated.
Could you please provide more details?

The APN payload you're sending to the device
The app state when the payload is sent (e.g., background, foreground, or inactive (app process completely terminated)
The iOS notification options you're using (e.g., badge, sound, count, critical, etc.)

I am currently unable to reproduce this issue.
The code you're referring to is triggered when a notification is received, but you're implying that the issue occurs when the push notification is activated.
Could you please provide more details?

The APN payload you're sending to the device
The app state when the payload is sent (e.g., background, foreground, or inactive (app process completely terminated)
The iOS notification options you're using (e.g., badge, sound, count, critical, etc.)

You are correct, the app is in the background, and the error occurs when the notification is received.
Payload is completely default as mentioned, 1:1 the example in the plugin
sound: default,
notId: number
content-available: 1
alert: {title: string, body: string}
Looking at the exception, it seems like the bug is caused by the dictionary beeing a regular dictionary
https://github.com/havesource/cordova-plugin-push/blob/12a5b71cb381c8e0ad0370a771ac523dc46de7a3/src/ios/PushPlugin.m#L36
but is cast like a mutable, which it is not
https://github.com/havesource/cordova-plugin-push/blob/12a5b71cb381c8e0ad0370a771ac523dc46de7a3/src/ios/PushPlugin.m#L497

Hi @erisu an update on this issue, its reproducible with this very minimum payload sent via Amazon SNS
Payload
  "APNS": "{\"aps\":{\"alert\": \"Sample message for iOS endpoints\", \"content-available\": \"1\"} }",
The culprit here is the content-available: 1 which triggers info copied to the notificationMessage, which then crashed when ran in the notificationReceived method, because notificationMessage is not actually a NSMutableDictionary but a NSDictonary
I tried creating a PR for this #315

Here is the following test case I set up and attempted to reproduce the issue, but I was still unable to do so.

Created a project on AWS Pinpoint for my test application (APNs Sandbox).

Initialized the plugin with the following:
document.addEventListener('deviceready', onDeviceReady, false);

function onDeviceReady() {
    console.log('Running cordova-' + cordova.platformId + '@' + cordova.version);
    document.getElementById('deviceready').classList.add('ready');

    const push = PushNotification.init({
        ios: {
            alert: true,
            badge: true,
            sound: true
        }
    });

    push.on('registration', onPushRegistration);
    push.on('notification', onPushNotification);
    push.on('error', onPushError);
}

Opened the application and placed it in the background.

Pushed a test message using AWS Pinpoint with the following content:
{
  "APNSMessage": {
    "aps": {
      "alert": "Sample message for iOS endpoints",
      "content-available": "1"
    }
  }
}

Results:

The push notification toast appeared.
Tapping on the toast opened the application as expected.
The application did not crash as reported.

Thoughts
Based on previous discussions, casting an NSDictionary as an NSMutableDictionary would be the issue you reported.
I reviewed the entire codebase to identify all instances where self.notificationMessage is set. Here's what I found:

didReceiveRemoteNotification

Stores userInfo, which is an NSDictionary.

pushPluginOnApplicationDidBecomeActive

Stores self.launchNotification, which is an NSDictionary.

willPresentNotification

Stores modifiedUserInfo, which is an NSMutableDictionary.

didReceiveNotificationResponse

Stores userInfo, which is an NSDictionary.
However, when content-available is set to 1 and notId is missing, it attempts to call setValue for the notId key on userInfo would not be correct because userInfo is immutable (an NSDictionary). To address this, userInfo should be made mutable (by calling [userInfo mutableCopy]), the notId should be set on the mutable copy, and then it should be copied back as an immutable dictionary.

notificationReceived

Clears self.notificationMessage by setting it to nil.

pushRegistry

Stores dictionaryPayload, which is an NSDictionary.

Proposed Solution
To ensure consistency and prevent crashes, I propose the following changes while keeping self.notificationMessage as an immutable NSDictionary (since most areas already store an NSDictionary into it):

didReceiveRemoteNotification:

Create a mutable copy of userInfo ([userInfo mutableCopy]), add notId if missing, and then store an immutable copy ([mutableUserInfo copy]) in self.notificationMessage.

willPresentNotification:

Ensure self.notificationMessage stores an immutable copy of modifiedUserInfo using [modifiedUserInfo copy].

notificationReceived:

When making changes (such as removing applicationState), create a mutable copy of self.notificationMessage ([self.notificationMessage mutableCopy]), make the changes, and then store it back as an immutable copy ([mutableNotificationMessage copy]).

Here is a commit I was working on but not tested yet: https://github.com/havesource/cordova-plugin-push/compare/master...havesource:cordova-plugin-push:fix/GH-314

I created PR #317. You should be able to test it and confirm whether it resolves the crash.
Could you also explain a little on how you handle content-available in your implementation?
During my testing, I noticed that the notification event handler is triggered twice. While I believe this is the original behavior, it feels undesirable, yet might only be resolved using flags.
Why it triggers twice:

When a push payload is delivered, the app receives it via the didReceiveRemoteNotification method. This method contains the content-available, slient push logic.

If the app is in the background or closed and content-available is set to 1 the notification is processed, results are sent to front-end, and then triggering the push notification event (JS).

If the user taps the notification toast, the didReceiveNotificationResponse method is invoked. This also processes the notification and sends results to the front-end, and triggering the same push notification event (JS).

Just for tests, if the JS notification event contains an alert, you’ll see the alert twice in this sequence:

once for didReceiveRemoteNotification when content-available is set to 1 and not in foreground.
again when the user taps the notification toast

Do you have any conditions in your notification handler to differentiate between background processing and when the toast is clicked? Such logic might help avoid duplication or redundant handling of the same notification on the front-end.
Here might be a rough diagram to see the flow?

Yes we are carefull not to handle the payload twice, here we rely on coldstart, background/foreground, and the notid. We save every notid, and mark it as handled (we send notid from backend).
There have been som issues with coldstart beeing set false or reverse on Android sometimes, so we have some workarounds for that.
We need the content-available:1 to fetch the data in the app, that the notification mentions.

Thank you for clarifying and confirming.
There’s an open issue ticket related to coldstart for iOS (#316), and I believe I have the solution. I’ll prepare a PR for it after this issue's PR is merge.
Let me know if you have time to test the PR and confirm whether it resolves everything for you.

I got the crash with v5 also.

Hi @erisu I have tried to test it locally using Xcode, both with the simulator which now support push testing, and my own device, but im not having much luck, as both endpoints seem to get disabled by SNS, when I send payloads, I don't know why, but maybe because of missing local project settings?
I have tried the new feature where you can drag n drop a json payload onto the device, and that works for the regular handlers of the plugin, but it does not activate the background content-available:1 handler, maybe because of missing functionality of the simulator.
But it seems like @geoidesic has tested it with succes, and the code changes seem on point, at both me and him are experiencing the issue with the same code, and the same line of the plugin.

Hi @sithwarrior
When I tested with Amazon SNS, I uploaded the P12 certificate specifically for debug builds. Apple provides a certificate that they state is suitable for both debug and release builds, but in my experience, the services I tested with in the past only recognized it for production use. I had to create the debug certificate.
I am not sure if this is due to a configuration issue, but that might be a possibility.
When testing from remote services, such as Amazon SNS, I believe you can only push notifications to physical devices.
While simulators might display an APN ID, I believe these IDs are not valid for remote pushes. You can only deliver a payload to the simulator via drag-and-drop or by using xcrun simctl push. However, these methods may not allow you to test all possible cases.

That's also my conclusion, that I have to specifically target the sandbox APNS, but that does require a bunch of trickery on my end to get going with my current build.


How to set Keep Alive Timeout: GCP kubernetes Load Balancer 502 issue

Hi,
How can set a keep alive timeout?
I've been checking the documentation and the code and I couldn't find anything.
My case:
We are deploying to Google Cloud with kubernetes. and we are getting random 502 responses because the GCP´s Load Balancer tries so keep connections open and looks like Cowboy closes the connection before the Load Balancer does. The Load Balancer sends requests to a closed connection.
Better explained here .
The solution for nginx is to set the keepalive_timeout to a bigger time than the Load Balancer.
Any Idea how to do it with Cowboy?.
Thanks in advance.
My current version is 1.1.2
With phoenix 1.3.2

In the first version of Cowboy it was a protocol option called timeout and it defaults to 5000ms.

@essen thanks for the fast response.
I've tried to set this option via phoenix config without luck.
I know is not is no related to this library but wouldn't you know how to pass the config from phoenix?.
I've tried:
config :my_app, MyApp.Endpoint,
  http: [port: {:system, "PORT"}, timeout: 30_000],
  #...

config :my_app, MyApp.Endpoint,
  http: [port: {:system, "PORT"}, protocol_options: [timeout: 30_000]],
  #...

config :my_app, MyApp.Endpoint,
  http: [port: {:system, "PORT"},cowboy: [timeout: 30_000]],
  #...

@arturictus The first option is the correct one. It is passed to ranch using https://github.com/elixir-plug/plug/blob/v1.5.0/lib/plug/adapters/cowboy.ex#L137

Try the correct option with wireshark or other to see what's happening. Just telnet would be good enough to confirm the timeout is used as expected.

The setup for Google Cloud Load Balancer should be:
config :my_app, MyApp.Endpoint,
  http: [port: {:system, "PORT"},
         timeout: 30_000,
         protocol_options: [max_keepalive: 10_000]],
  #...

I hope that helps

@arturictus Google specifies in their docs that it should be set greater than 600 seconds for the keepalive, is there a reason why you say it should be 10 seconds https://cloud.google.com/load-balancing/docs/https/#timeouts_and_retries

Hi @michaelst ,
https://ninenines.eu/docs/en/cowboy/1.0/guide/http_req_life/

Cowboy will only accept a certain number of new requests on the same connection. By default it will run up to 100 requests. This number can be changed by setting the max_keepalive configuration value when starting an HTTP listener.

but you are right, I should bump the timeout to 600_000
corrected setup:
config :my_app, MyApp.Endpoint,
  http: [port: {:system, "PORT"},
         timeout: 605_000,
         protocol_options: [max_keepalive: 10_000]],
  #...

That is incorrect also as it is not the timeout but the keepalive. These are the settings that finally stopped the 502 errors for us
  https: [
    port: 443,
    timeout: 30_000,
    protocol_options: [max_keepalive: 600_001],
  ],

Hi @michaelst,
I really don't get it how.
The google documentation says:

TCP session timeout, whose value is fixed at 10 minutes (600 seconds)

The cowboy documentation says about max_keepalive:

https://ninenines.eu/docs/en/cowboy/1.0/manual/cowboy_protocol/
Maximum number of requests allowed per connection.

This comes from Plug.Adapter.Cowboy: https://hexdocs.pm/plug/Plug.Adapters.Cowboy.html
How number of request affect to the timeout?

I was reading the settings as max_keepalive being the keepalive timeout for a connection and timeout being for an individual request. It also appears to behave this way as we have not had any 502 errors since changing to the configuration I posted where we did have 502 errors with your revised configuration. I need to look into the documentation some more to see what is happening.

A TCP session timeout, whose value is fixed at 10 minutes (600 seconds). This session timeout is sometimes called a keepalive or idle timeout

max_keepalive is not a timeout, it's the max number of requests on the same connection using the keepalive mechanism. It's possible you get 502s both when Cowboy timeouts and when Cowboy closes the connection when that number is reached though. I suppose infinity would be a better value for it in that case.

Any further progress on this issue, trying to solve it myself for GCP and it seems like setting max_keepalive to 600_001 is a bandaid until you have that number of user requests in 10 minute intervals?

There's no issue as far as I can tell.

Hi all, I don't think there's an issue—maybe a lack of documentation? This one is still giving me headaches, and I think it's terminology related. As @michaelst points out on the GCP Kubernetes docs:
"""
A TCP session timeout, whose value is fixed at 10 minutes (600 seconds). This session timeout is sometimes called a keepalive or idle timeout, and its value is not configurable by modifying your backend service. You must configure the web server software used by your backends so that its keepalive timeout is longer than 600 seconds to prevent connections from being closed prematurely by the backend. This timeout does not apply to WebSockets.
"""
If I'm reading that right, and looking through the docs here (https://ninenines.eu/docs/en/cowboy/2.8/manual/cowboy_http/), I think the key setting is idle_timeout, and not timeout. I'm trying this:
config :my_ap, MyAppWeb.Endpoint,
  http: [
    protocol_options: [
      # https://ninenines.eu/docs/en/cowboy/2.8/manual/cowboy_http/
      request_timeout: 30_000,
      # https://cloud.google.com/load-balancing/docs/https/#timeouts_and_retries
      idle_timeout: 650_000, # must be greater than 600s
      max_keepalive: :infinity
    ]
  ],
  render_errors: [accepts: ~w(html json)],
  pubsub_server: MyApp.PubSub

Will update here if it works.
Has anyone else solved this?

timeout was in Cowboy 1.x, it's called idle_timeout now, so I think your configuration is correct.

Thank you! Yes, seems to be working for me, btw.


(with Poison 2.2) ** (Poison.EncodeError) expected string or atom key, got: 50

Seems I finally found one of the differences of poison 3 vs. poison 2.
The problem is our new percentiles that have integers as keys in the maps:
iex(1)> Poison.encode! %{1 => "lol"}
** (Poison.EncodeError) expected string or atom key, got: 1
    (poison) lib/poison/encoder.ex:188: Poison.Encoder.Map.encode_name/1
    (poison) lib/poison/encoder.ex:212: anonymous fn/4 in Poison.Encoder.Map.encode/3
    (poison) lib/poison/encoder.ex:214: Poison.Encoder.Map."-encode/3-lists^foldl/2-0-"/3
    (poison) lib/poison/encoder.ex:214: Poison.Encoder.Map.encode/3
    (poison) lib/poison.ex:41: Poison.encode!/2

vs.
iex(1)> Poison.encode! %{1 => "lol"}
"{\"1\":\"lol\"}"

This should trigger in benchee_json, but right now this code is still inlined in benchee_html:
job_json = JSON.encode!(%{
        statistics: scenario.run_time_statistics,
        run_times: scenario.run_times
      })

We can't switch to a newer poison version as lots of people rely on Poison < 3 in some phoenix versions etc.
Mitigation:

manually stringify keys
switch to a different json library that handles these things like jason (not sure if it handles them/worth a try)

I think now that the benchee_json PR is merged, this can be closed, right?

Yup should be! :)


Add without_vsync or something similar

Add the possibility to ask not to have vsync when creating the window.
This can be without_vsync, or by passing a parameter to with_vsync.

This is a very cute outdated issue.


[do not merge] Revert "Fix regression bug caused by sharing TSerializer across multiple threads"

Reverts apache/pinot#8160

Codecov Report

Merging #8179 (e5746b0) into master (5fa4737) will decrease coverage by 57.23%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             master    #8179       +/-   ##
=============================================
- Coverage     71.41%   14.17%   -57.24%
+ Complexity     4302       81     -4221
=============================================
  Files          1623     1578       -45
  Lines         84312    82430     -1882
  Branches      12639    12435      -204
=============================================
- Hits          60213    11687    -48526
- Misses        19974    69870    +49896
+ Partials       4125      873     -3252

Flag
Coverage Δ

integration1
?

integration2
?

unittests1
?

unittests2
14.17% <0.00%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5fa4737...e5746b0. Read the comment docs.


more new features

Codecov Report
Base: 94.57% // Head: 94.24% // Decreases project coverage by -0.33% :warning:

Coverage data is based on head (4e44875) compared to base (31955ee).
Patch coverage: 75.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #19      +/-   ##
==========================================
- Coverage   94.57%   94.24%   -0.34%
==========================================
  Files           8        8
  Lines         461      469       +8
==========================================
+ Hits          436      442       +6
- Misses         25       27       +2

Impacted Files
Coverage Δ

typed_argparse/app.py
95.52% <75.00%> (-1.31%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Loss of precision on int64s in numpy >= 1.22

import fixfmt
import numpy
fmt = fixfmt._ext.Number(19, -1, pad=' ', sign=' ' , nan='NaN', inf='inf', point='.', bad='#')
x =<PHONE_NUMBER>000000127
print(fmt(x))
#<PHONE_NUMBER>000000127
print(fmt(numpy.int64(x)))
#<PHONE_NUMBER>000000000

numpy 1.22.0 and up all reveal this issue, but 1.21.6 does not.

Reproduced; thanks for reporting.  I'll take a look.
Happy to see someone using this repo.

Fixed in 0.13.3.

Thank you!


session? -> cloudSession!

In the code example in this section: https://docs.microsoft.com/en-us/azure/spatial-anchors/how-tos/create-locate-anchors-unity#provide-frames-to-the-session
it mentions "session". It's very unclear what you mean with session, but it seems you mean what is called this.cloudSession in the other code snippets?
It would be super helpful if the information on this page was put into one clear code example, saves time and minimizes unclarity.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 47ed5367-ee97-31a2-78ff-dd98e5c23d24
Version Independent ID: 2e975197-ca07-351a-f538-372500c0cc8f
Content: Create & locate anchors in Unity - Azure Spatial Anchors
Content Source: articles/spatial-anchors/how-tos/create-locate-anchors-unity.md
Service: azure-spatial-anchors
GitHub Login: @RamonArguelles
Microsoft Alias: rgarcia

@jelmerS2  Thank you for your query, our team will look into it and get back to you at the earliest.

Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

Hi @jelmerS2, thanks for reporting this, we will refresh our doc to be unambiguous, I've filed an internal bug [#26543435] to track.

The definition of the "session" is made clear in the first sentence of the paragraph where we describe the "spatial anchor session". We'll consider this feedback in a future update.
You can also find examples of all of these concepts in our samples.
@asergaz Close?

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Genericize the SlmQueue project into the non-ZF bits and the ZF-required bits for broader adoption

Currently, the implementation of SlmQueue appears tightly coupled to the whole ZF Event Manager side of things. Making this project generic would make these abstractions more useful across a broader set of PHP projects (e.g., very lean PSR-7-driven APIs).
Is this something that's currently under consideration for this project at this point in time?
My initial thoughts would be to expose interfaces that must be implemented (similar to how League approaches the OAuth2 implementation) -- example project: https://oauth2.thephpleague.com/
The benefit of this approach is that the integration details could be exposed as interfaces and direct attachment into something like ZF would be an event-manager binding mechanism ... but for other projects one could implement their own bindings as it is appropriate.

It's not considered at the moment, but your points are valid. That said it is possible to use this module outside an zend framework MVC application. Myself is using it in an expressive application without any problems. I only needed to recreate a cli command to start the doctrine queue.
I would like to see a more generic v2 version. PHP 7 only. Unfortunately I simply don't have the time to do that... About refactoring to something that doesn't have events I not so sure. We wanted to make something evented and the zf event manager was a natural fit and a good tool available within the zf eco system. It got the job done. My current application doesn't use the zf event manager any other way, but has it's own implementation for the most part. Do we care?

Well, to be honest ... I've gone away from the event-driven system style of development for much of my backend development these days. I find being more explicit in my code has led to more sane debugging processes and provides for more coherence for folks who are also picking up a new microservice I'm working on or otherwise. So to me the Event-driven side of things is less than useful / ideal. Others may disagree, of course ... but after working in the WP ecosystem which is events on steroids IMHO ... I no longer wish to have much to do with events unless I have a very specific business case for doing so.

I agree, events can be hell to debug. But to change something that works for the sake of change, no at this time.
However I encourage you to come up with a different solution if you require one for v2... :-)

@basz maybe you can share code how you implemented this in zend-expressive? would be great help :) its not that simple as just writing console commands. I hope you did this with symfony/console but that's not that important.

https://gist.github.com/basz/6411860cd72404543e39a325ece328e5

I think that's it. Scanning my source quickly I don't believe you need to do much more (besides adding the config to autoload). Ofcourse the controller plugins won't work

hmm what you did with this https://github.com/juriansluiman/SlmQueue/blob/a4f09a1c4dece7fd2d94498249affb8dea7e07b1/src/Factory/WorkerFactory.php#L25 this event manager is configured in zend-mvc but I guess we could just do new EventManager() no or I will have to create alias for this?

i see what you mean;
I have this in my config/container.php
$config      = require __DIR__ . '/config.php';
$container   = new ServiceManager($config['dependencies']);
$mvcServices = new ServiceManagerConfig();

$mvcServices->configureServiceManager($container);
$container->setAlias('Config', 'config');
$container->setAlias('Configuration', 'config');
$container->setService('config', $config);

return $container;

so I have to have zend-mvc as dependancy? :(

I'm sorry I knee deep in some other stuff; I guess what you need is an implementation of EventManagerInterface. I don't think it needs to come from the (mvc) service locator. There is no event messaging outside of the packages. new EventManager should do. Are you planning on removing the dependency? In my current project I just live with it even while its a expressive app and nothing else uses it...

I am thinking maybe its possible to do it in this module create new eventmanager instead of getting it from service manager would it introduce any BC break?

I think in general it is not a problem and only a breaking change for anyone who is listening to events through the 'global' event listener.
Not sure who or why does that

I have created PR for this https://github.com/juriansluiman/SlmQueue/pull/201


Update location of particles.py

A token of my appreciation for this code.

Much appreciated!

Much appreciated!


FFMpegWriter input and output dict format

I installed the most recent version of scikit-video, and I was wondering if there is an example format for the inputdict or outputdict formats or keys.  Going through the tutorial at (http://www.scikit-video.org/stable/io.html) the given example of this code doesn't work as written with the current version of scikit-video:
writer = skvideo.io.FFmpegWriter("outputvideo.mp4", (5, 480, 640, 3))
Is there a way an example could be provided of what the inputdict and outputdict values would be to alter the parameters of FFmpegWriter?

Ah good catch. I will have to update the documentation about FFmpegWriter. I'm in the process of writing more example-based documentation. You can see the current progress (with dict examples for I/O) here:
http://www.scikit-video.org/dev/examples.html

Hey guys. Check this code :
from skvideo.io import VideoWriter
import numpy
writer = VideoWriter(filename, frameSize=(w, h))
writer.open()
writer.write("some random video frame")
It does output a video file, however the quality of it is very poor beyond the first keyframe. Your skvideo documentation states that FFmpegWriter should be used for more customisation of parameters, however that does not exist.
http://www.scikit-video.org/dev/examples.html this link does not provide an example of video file written to disk frame by frame.
What's wrong with this thread? Why is there such bad quality of output? Why is there such bad quality of documentation?
Why is there no proper way to write video in Python, in 2016? :)

Ah! I did forget to add an example on the webpage regarding high-quality h264 encodes.
If you want to output a good quality video (frame by frame), you can start from the below example which uses the libx264 codec with bitrate set to 30MBps which encodes random noise:
import skvideo.io
import skvideo.measure
outputfile = sys._getframe().f_code.co_name + ".mp4"
outputdata = np.random.random(size=(30, 480, 640, 3)) * 255
outputdata = outputdata.astype(np.uint8)

# start the FFmpeg writing subprocess with following parameters
writer = skvideo.io.FFmpegWriter(outputfile, outputdict={
    '-vcodec': 'libx264', '-b': '30000000'
})

for i in range(30):
    writer.writeFrame(outputdata[i])
writer.close()

inputdata = skvideo.io.vread(outputfile)

# test each frame's SSIM score
mSSIM = 0
for i in range(30):
    mSSIM += skvideo.measure.ssim(np.mean(inputdata[i], axis=2), np.mean(outputdata[i], axis=2))
mSSIM /= 30.0

print mSSIM

You can play with the coding parameters and see how it is reflected in the measured mean SSIM value. BTW, the flags in the outputdict are passed directly to FFmpeg through the CLI.
Perhaps there is a better, more generic way to write high quality video as h264, rather than explicitly using a dictionary of parameters. I think the current approach is flexible although currently poorly documented.

I have now added this to the dev branch of the website

It would be awesome if you could state somewhere whether it's possible to grab frames from a webcam? Have not been able to find any documentation on that in the latest release, and had no luck with ffmpegreader so far..

What have you tried, and what platform are you using? If on Linux, I think
it could work if you point the input file to /dev/video0, but I haven't
tried that.
On Nov 22, 2017 8:23 PM, "Tobi"<EMAIL_ADDRESS>wrote:
It would be awesome if you could state somewhere whether it's possible to
grab frames from a webcam? Have not been able to find any documentation on
that in the latest release, and had no luck with ffmpegreader so far..
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/scikit-video/scikit-video/issues/11#issuecomment-346519467,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIAHzRytaPzGQh3XMxDMhCjpWZk3IobNks5s5NcpgaJpZM4IQL0y
.

Hi,
Thanks for the ice package!
I am working with skvideo and trying to output a video but while the code above (high quality video) runs and produces a video file, the created video is not being able to be read by Window media player/ does not have any frames. Do you know what is causing the problem and how could I fix this?

Is the file size larger than 0? You can try playing it with VLC

Hi, I downloaded VLC and it works perfectly! Thank you.

Until today we don't have a simple documentation for outputdict from FFmpegWriter. Is one of the most important things on this lib. How can I define how to write an video? Nowhere with explanations? A single example?


Support for AVITI Run Manifest

Add 'Download AVITI Run Manifest' button to generate attached CSV file.
Run#-PEGRid#,Index1,Reverse-Complement Index2,Lane
Lane options are 1, 2, or 1+2

Header of the file should be:
[SETTINGS],,,,
SettingName,Value,Lane,,
SpikeInAsUnassigned,FALSE,,,
# Lines can be commented out with a leading # sign. ,,,,
# Read mask is set to all cycles less the last imaged (which is used for calculations),,,,
R1FastQMask,R1:Y*,1+2,,
R2FastQMask,R2:Y*,1+2,,
,,,,
# Index mask is set to default and FASTQ  not generated for Lanes 1 and 2.,,,,
I1Fastq,FALSE,1+2,,
I1Mask,I1:Y8N*,1+2,,
I1MismatchThreshold,1,1+2,,
I2Fastq,FALSE,1+2,,
I2MismatchThreshold,1,1+2,,
I2Mask,I2:Y8N*,1+2,,
,,,,
# UMI mask is set to nothing with no FASTQ generated for Lanes 1 and 2.,,,,
UmiMask,I1:N*, 1+2,,
UmiFastQ,FALSE,1+2,,
,,,,
# You can optionally trim adapters. The Elevate adapter is known so no need to specify.  In this template adapter trimming is turned off. ,,,,
AdapterTrimType, Paired-End, 1+2,,
R1AdapterTrim,FALSE,1+2,,
R1AdapterNMask,FALSE,1+2,,
R2AdapterTrim,FALSE,1+2,,
R2AdapterNMask,FALSE,1+2,,
,,,,
[RunValues],,,,
Keyname, Value,,,
Example-Custom-Key,Additional metadata can be added as a key-value pair.,,,
,,,,
[SAMPLES],,,,
SampleName,Index1,Index2,Lane,Project
# Make sure that in the Index1 and Index2 columns the index sequences for each library is in the same orientation (samples and PhiX controls).,,,,
Adept_CB1,ATGTCGCT,CTAGCTCG,1+2
Adept_CB2,CACAGATC,ACGAGAGT,1+2
Adept_CB3,GCACATAG,GACTACTA,1+2
Adept_CB4,TGTGTCGA,TGTCTGAC,1+2

Index2 column should be reverse-complemented

def reverseComplement(String sequence) {
    def complementMap = [A: 'T', T: 'A', C: 'G', G: 'C']
    def reversedSequence = sequence.reverse().collect { nucleotide ->
        complementMap.containsKey(nucleotide as char) ? complementMap[nucleotide as char] : nucleotide
    }.join()
    return reversedSequence
}

def dnaSequence = "ATCGATCG"
def reverseComplementedSequence = reverseComplement(dnaSequence)
println("Original sequence: $dnaSequence")
println("Reverse-complemented sequence: $reverseComplementedSequence")