Registrant not consistent for a few TLDs

The registrant sometimes is not the real one. Let's take google as an example. Here the list of there domains: https://www.google.com/supported_domains
>>> import requests
>>> r = requests.get("https://www.google.com/supported_domains")
>>> domains = [domain[1:] for domain in r.text.splitlines()]
>>> registrants = []
>>> for domain in domains:
...     try:
...        w = whoisdomain.query(domain)
...     except Exception as e:
...        print(e)
...        continue
...     if hasattr(w, "registrant"):
...        if registrants.get(w.registrant):
...           registrants[w.registrant].append(domain)
...        else:
...           registrants[w.registrant] = [domain, ]

Here the results:
{None: ['google.com.ag',
        'google.as',
        'google.bg',
        'google.com.bo',
        'google.cd',
        'google.fi',
        'google.ge',
        'google.gg',
        'google.hr',
        'google.im',
        'google.je',
        'google.kg',
        'google.kz',
        'google.com.ly',
        'google.co.ma',
        'google.mg',
        'google.com.mx',
        'google.com.ng',
        'google.nu',
        'google.ro',
        'google.se',
        'google.sn',
        'google.sm',
        'google.td',
        'google.com.tw',
        'google.co.ug',
        'google.ws',
        'google.rs'],
 '': ['google.ae',
      'google.com.ar',
      'google.com.au',
      'google.bf',
      'google.com.br',
      'google.de',
      'google.dk',
      'google.fm',
      'google.gl',
      'google.co.id',
      'google.ie',
      'google.co.il',
      'google.co.jp',
      'google.la',
      'google.lt',
      'google.lu',
      'google.lv',
      'google.mu',
      'google.nl',
      'google.no',
      'google.com.om',
      'google.pl',
      'google.pt',
      'google.com.qa',
      'google.ru',
      'google.tm',
      'google.com.ua',
      'google.co.uk',
      'google.co.za'],
 'ADMIN-LEO': ['google.co.ls'],
 'CN_10': ['google.co.cr'],
 'CON000020360': ['google.co.ve'],
 'G830057': ['google.si'],
 'GDA-ITFARM': ['google.co.tz'],
 'GI7803022-NICAT': ['google.at'],
 'GL210-IS': ['google.is'],
 'GOOGLE INC': ['google.bj'],
 'GOOGLE LLC (SGNIC-ORG1624232)': ['google.com.sg'],
 'Google Canada Corporation': ['google.ca'],
 'Google Inc.': ['google.co.zm'],
 'Google Ireland Holdings Unlimited Company': ['google.fr', 'google.it'],
 'Google Korea, LLC': ['google.co.kr'],
 'Google LLC': ['google.com',
                'google.com',
                'google.am',
                'google.bi',
                'google.by',
                'google.ci',
                'google.cl',
                'google.cm',
                'google.com.co',
                'google.dm',
                'google.ee',
                'google.com.gi',
                'google.co.in',
                'google.co.ke',
                'google.com.lb',
                'google.me',
                'google.mn',
                'google.co.mz',
                'google.com.na',
                'google.com.pe',
                'google.com.pr',
                'google.rw',
                'google.com.sa',
                'google.sc',
                'google.sh',
                'google.com.sl',
                'google.so',
                'google.st',
                'google.tn',
                'google.com.tr',
                'google.com.vc',
                'google.cat'],
 'Google LLC (กูเกิล แอลแอลซี)': ['google.co.th'],
 'HONG KONG INTERNET HOLDING LIMITED': ['google.com.hk'],
 'MM1171195': ['google.cz'],
 'Not shown, please visit www.dnsbelgium.be for webbased whois.': ['google.be'],
 'Techno Bros. IT Solution Pty. Ltd.': ['google.com.et'],
 'UNET-R11': ['google.mk'],
 'mmr-170347': ['google.sk'],
 '北京谷翔信息技术有限公司': ['google.cn']}

Example of wrong values:

'MM1171195': ['google.cz'],
'UNET-R11': ['google.mk'],
'mmr-170347': ['google.sk'],
'ADMIN-LEO': ['google.co.ls'],
'CN_10': ['google.co.cr'],
'CON000020360': ['google.co.ve'],
'G830057': ['google.si'],
'GDA-ITFARM': ['google.co.tz'],
'GI7803022-NICAT': ['google.at'],
'GL210-IS': ['google.is'],
'' and None for a few of them

It could be easily fixed with the most of them by using Registrant Organization like in those examples below:
❯ whois google.cz
%  (c) 2006-2021 CZ.NIC, z.s.p.o.
%
% Intended use of supplied data and information
%
% Data contained in the domain name register, as well as information
% supplied through public information services of CZ.NIC association,
% are appointed only for purposes connected with Internet network
% administration and operation, or for the purpose of legal or other
% similar proceedings, in process as regards a matter connected
% particularly with holding and using a concrete domain name.
%
% Full text available at:
% http://www.nic.cz/page/306/intended-use-of-supplied-data-and-information/
%
% See also a search service at http://www.nic.cz/whois/
%
%
% Whoisd Server Version: 3.12.2
% Timestamp: Wed Aug 30 13:06:30 2023

domain:       google.cz
registrant:   MM1171195
admin-c:      MM1171195
nsset:        MM1543911
registrar:    REG-MARKMONITOR
registered:   21.07.2000 15:21:00
changed:      23.04.2018 20:24:01
expire:       22.07.2024

contact:      MM1171195
org:          Google LLC
name:         Domain Administrator
address:      1600 Amphitheatre Parkway
address:      Mountain View
address:      94043
address:      CA
address:      US
registrar:    REG-MARKMONITOR
created:      02.03.2018 18:52:05
changed:      15.05.2018 21:32:00

nsset:        MM1543911
nserver:      ns2.google.com
nserver:      ns4.google.com
nserver:      ns3.google.com
nserver:      ns1.google.com
tech-c:       MM193020
registrar:    REG-MARKMONITOR
created:      18.05.2011 23:27:16

contact:      MM193020
org:          MarkMonitor Inc.
name:         Domain Provisioning
address:      2150 S Bonito Way
address:      Suite 150
address:      Meridian
address:      83642
address:      ID
address:      US
registrar:    REG-MARKMONITOR
created:      03.02.2011 18:24:34
changed:      29.06.2021 23:29:20

or
❯ whois google.sk
Domain:                       google.sk
Created:                      2003-07-24
Valid Until:                  2024-07-24
Updated:                      2023-06-22
Domain Status:                clientTransferProhibited, clientUpdateProhibited, clientDeleteProhibited
Nameserver:                   ns1.google.com
Nameserver:                   ns2.google.com
Nameserver:                   ns3.google.com
Nameserver:                   ns4.google.com

Domain registrant:            mmr-170347
Name:                         Domain Administrator
Organization:                 Google Ireland Holdings Unlimited Company
Organization ID:              369511
Phone:                        +353.14361000
Email<EMAIL_ADDRESS>Street:                       70 Sir John Rogerson's Quay
City:                         Dublin
Postal Code:                  2
Country Code:                 IE
Authorised Registrar:         MARK-0292
Created:                      2019-06-07
Updated:                      2019-06-07

Registrar:                    MARK-0292
Name:                         MarkMonitor International Limited
Organization:                 MarkMonitor International Limited
Organization ID:              4847541
Phone:                        +1.2083895740
Email<EMAIL_ADDRESS>Street:                       12 New Fetter Lane
City:                         London
Postal Code:                  EC4A 1JP
Country Code:                 UK
Created:                      2018-06-27
Updated:                      2023-08-03

Administrative Contact:       mmr-170347
Name:                         Domain Administrator
Organization:                 Google Ireland Holdings Unlimited Company
Organization ID:              369511
Phone:                        +353.14361000
Email<EMAIL_ADDRESS>Street:                       70 Sir John Rogerson's Quay
City:                         Dublin
Postal Code:                  2
Country Code:                 IE
Created:                      2019-06-07
Updated:                      2019-06-07

Technical Contact:            mmr-170347
Name:                         Domain Administrator
Organization:                 Google Ireland Holdings Unlimited Company
Organization ID:              369511
Phone:                        +353.14361000
Email<EMAIL_ADDRESS>Street:                       70 Sir John Rogerson's Quay
City:                         Dublin
Postal Code:                  2
Country Code:                 IE
Created:                      2019-06-07
Updated:                      2019-06-07

what exactly is wrong with  ./test2.py -d google.cz
test domain: <<<<<<<<<< google.cz >>>>>>>>>>>>>>>>>>>>
name               str               'google.cz'
tld                str               'cz'
registrar          str               'REG-MARKMONITOR'
registrant_country str               ''
creation_date      datetime.datetime 2000-07-21 15:21:00
expiration_date    datetime.datetime 2024-07-22 00:00:00
last_updated       datetime.datetime 2018-04-23 20:24:01
dnssec             bool              False
status             str               ''
statuses           list              []
name_servers       list              ['ns1.google.com', 'ns2.google.com', '
ns3.google.com', 'ns4.google.com']
*registrant         str               'MM1171195'*emails             list
[]
On Wed, Aug 30, 2023 at 11:36 PM Badreddine Lejmi @.***>
wrote:

The registrant sometimes is not the real one. Let's take google as an
example. Here the list of there domains:
https://www.google.com/supported_domains

import requests
r = requests.get("https://www.google.com/supported_domains")
domains = [domain[1:] for domain in r.text.splitlines()]
registrants = []
for domain in domains:
...     try:
...        w = whoisdomain.query(domain)
...     except Exception as e:
...        print(e)
...        continue
...     if hasattr(w, "registrant"):
...        if registrants.get(w.registrant):
...           registrants[w.registrant].append(domain)
...        else:
...           registrants[w.registrant] = [domain, ]

Here the results:
{None: ['google.com.ag',
'google.as',
'google.bg',
'google.com.bo',
'google.cd',
'google.fi',
'google.ge',
'google.gg',
'google.hr',
'google.im',
'google.je',
'google.kg',
'google.kz',
'google.com.ly',
'google.co.ma',
'google.mg',
'google.com.mx',
'google.com.ng',
'google.nu',
'google.ro',
'google.se',
'google.sn',
'google.sm',
'google.td',
'google.com.tw',
'google.co.ug',
'google.ws',
'google.rs'],
'': ['google.ae',
'google.com.ar',
'google.com.au',
'google.bf',
'google.com.br',
'google.de',
'google.dk',
'google.fm',
'google.gl',
'google.co.id',
'google.ie',
'google.co.il',
'google.co.jp',
'google.la',
'google.lt',
'google.lu',
'google.lv',
'google.mu',
'google.nl',
'google.no',
'google.com.om',
'google.pl',
'google.pt',
'google.com.qa',
'google.ru',
'google.tm',
'google.com.ua',
'google.co.uk',
'google.co.za'],
'ADMIN-LEO': ['google.co.ls'],
'CN_10': ['google.co.cr'],
'CON000020360': ['google.co.ve'],
'G830057': ['google.si'],
'GDA-ITFARM': ['google.co.tz'],
'GI7803022-NICAT': ['google.at'],
'GL210-IS': ['google.is'],
'GOOGLE INC': ['google.bj'],
'GOOGLE LLC (SGNIC-ORG1624232)': ['google.com.sg'],
'Google Canada Corporation': ['google.ca'],
'Google Inc.': ['google.co.zm'],
'Google Ireland Holdings Unlimited Company': ['google.fr', 'google.it'],
'Google Korea, LLC': ['google.co.kr'],
'Google LLC': ['google.com',
'google.com',
'google.am',
'google.bi',
'google.by',
'google.ci',
'google.cl',
'google.cm',
'google.com.co',
'google.dm',
'google.ee',
'google.com.gi',
'google.co.in',
'google.co.ke',
'google.com.lb',
'google.me',
'google.mn',
'google.co.mz',
'google.com.na',
'google.com.pe',
'google.com.pr',
'google.rw',
'google.com.sa',
'google.sc',
'google.sh',
'google.com.sl',
'google.so',
'google.st',
'google.tn',
'google.com.tr',
'google.com.vc',
'google.cat'],
'Google LLC (กูเกิล แอลแอลซี)': ['google.co.th'],
'HONG KONG INTERNET HOLDING LIMITED': ['google.com.hk'],
'MM1171195': ['google.cz'],
'Not shown, please visit www.dnsbelgium.be for webbased whois.': ['google.be'],
'Techno Bros. IT Solution Pty. Ltd.': ['google.com.et'],
'UNET-R11': ['google.mk'],
'mmr-170347': ['google.sk'],
'北京谷翔信息技术有限公司': ['google.cn']}
Wrong values:

'MM1171195': ['google.cz'],
'UNET-R11': ['google.mk'],
'mmr-170347': ['google.sk'],
'ADMIN-LEO': ['google.co.ls'],
'CN_10': ['google.co.cr'],
'CON000020360': ['google.co.ve'],
'G830057': ['google.si'],
'GDA-ITFARM': ['google.co.tz'],
'GI7803022-NICAT': ['google.at'],
'GL210-IS': ['google.is'],
'' and None for a few of them

It could be easily fixed with the most of them by using Registrant
Organization like in those examples below:
❯ whois google.cz
%  (c) 2006-2021 CZ.NIC, z.s.p.o.
%
% Intended use of supplied data and information
%
% Data contained in the domain name register, as well as information
% supplied through public information services of CZ.NIC association,
% are appointed only for purposes connected with Internet network
% administration and operation, or for the purpose of legal or other
% similar proceedings, in process as regards a matter connected
% particularly with holding and using a concrete domain name.
%
% Full text available at:
% http://www.nic.cz/page/306/intended-use-of-supplied-data-and-information/
%
% See also a search service at http://www.nic.cz/whois/
%
%
% Whoisd Server Version: 3.12.2
% Timestamp: Wed Aug 30 13:06:30 2023
domain:       google.cz
registrant:   MM1171195
admin-c:      MM1171195
nsset:        MM1543911
registrar:    REG-MARKMONITOR
registered:   21.07.2000 15:21:00
changed:      23.04.2018 20:24:01
expire:       22.07.2024
contact:      MM1171195
org:          Google LLC
name:         Domain Administrator
address:      1600 Amphitheatre Parkway
address:      Mountain View
address:      94043
address:      CA
address:      US
registrar:    REG-MARKMONITOR
created:      02.03.2018 18:52:05
changed:      15.05.2018 21:32:00
nsset:        MM1543911
nserver:      ns2.google.com
nserver:      ns4.google.com
nserver:      ns3.google.com
nserver:      ns1.google.com
tech-c:       MM193020
registrar:    REG-MARKMONITOR
created:      18.05.2011 23:27:16
contact:      MM193020
org:          MarkMonitor Inc.
name:         Domain Provisioning
address:      2150 S Bonito Way
address:      Suite 150
address:      Meridian
address:      83642
address:      ID
address:      US
registrar:    REG-MARKMONITOR
created:      03.02.2011 18:24:34
changed:      29.06.2021 23:29:20
or
❯ whois google.sk
Domain:                       google.sk
Created:                      2003-07-24
Valid Until:                  2024-07-24
Updated:                      2023-06-22
Domain Status:                clientTransferProhibited, clientUpdateProhibited, clientDeleteProhibited
Nameserver:                   ns1.google.com
Nameserver:                   ns2.google.com
Nameserver:                   ns3.google.com
Nameserver:                   ns4.google.com
Domain registrant:            mmr-170347
Name:                         Domain Administrator
Organization:                 Google Ireland Holdings Unlimited Company
Organization ID:              369511
Phone:                        +353.14361000
Email:                        @.***
Street:                       70 Sir John Rogerson's Quay
City:                         Dublin
Postal Code:                  2
Country Code:                 IE
Authorised Registrar:         MARK-0292
Created:                      2019-06-07
Updated:                      2019-06-07
Registrar:                    MARK-0292
Name:                         MarkMonitor International Limited
Organization:                 MarkMonitor International Limited
Organization ID:              4847541
Phone:                        +1.2083895740
Email:                        @.***
Street:                       12 New Fetter Lane
City:                         London
Postal Code:                  EC4A 1JP
Country Code:                 UK
Created:                      2018-06-27
Updated:                      2023-08-03
Administrative Contact:       mmr-170347
Name:                         Domain Administrator
Organization:                 Google Ireland Holdings Unlimited Company
Organization ID:              369511
Phone:                        +353.14361000
Email:                        @.***
Street:                       70 Sir John Rogerson's Quay
City:                         Dublin
Postal Code:                  2
Country Code:                 IE
Created:                      2019-06-07
Updated:                      2019-06-07
Technical Contact:            mmr-170347
Name:                         Domain Administrator
Organization:                 Google Ireland Holdings Unlimited Company
Organization ID:              369511
Phone:                        +353.14361000
Email:                        @.***
Street:                       70 Sir John Rogerson's Quay
City:                         Dublin
Postal Code:                  2
Country Code:                 IE
Created:                      2019-06-07
Updated:                      2019-06-07
—
Reply to this email directly, view it on GitHub
https://github.com/mboot-github/WhoisDomain/issues/21, or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7CCKLAN7STUIOKDZ7YM3CDXX6W3ZANCNFSM6AAAAAA4FEDTGU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

The registrant IMHO should be "Google Ireland Holdings Unlimited Company" preferably over "MM1171195"

That will need a sophisticaded regex to find the org string in the
registrar section , alternatively you could use cli whois to directly query
for the registrar handle once you have it
It is also a significant change from the original behaviour.
I expect due to gdpr rules some european whois tld responses may not have
the name string onky the handle.
On Thu, Aug 31, 2023, 17:13 Badreddine Lejmi @.***>
wrote:

The registrant IMHO should be "Google Ireland Holdings Unlimited Company"
preferably over "MM1171195"
—
Reply to this email directly, view it on GitHub
https://github.com/mboot-github/WhoisDomain/issues/21#issuecomment-1701237632,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7CCKLDHAL5BGKU34MZLHNLXYCSZDANCNFSM6AAAAAA4FEDTGU
.
You are receiving this because you commented.Message ID:
@.***>

I can handle this PR. It will not be so hard if I use the same technique as I did for .fr
Regarding GDPR, it's the organization's name, not the person's name so not it does not apply.
And even if it does apply, here the data processor is the user not the tool in itself.
The only issue a user may have with this tool, and that already exists actually, is the persistent file storage cache mechanism yet it's unrelated to this specific issue.

i was thinking of adding a group extract, extract a section of data
starting with a line registrar: untill the next empty line, and then we
could apply further regexes on the extracted section
but that may be messy in the code and i would have to add a
apply Regex B after extract A kind of behaviour
On Thu, Aug 31, 2023 at 8:48 PM Badreddine Lejmi @.***>
wrote:

I can handle this PR. It will not be so hard if I use the same technique
as I did for .fr
Regarding GDPR, it's the organization's name, not the person's name so not
it does not apply.
And even if it does apply, here the data processor is the user not the
tool in itself.
The only issue a user may have with this tool, and that already exists
actually, is the persistent file storage cache mechanism yet it's unrelated
to this specific issue.
—
Reply to this email directly, view it on GitHub
https://github.com/mboot-github/WhoisDomain/issues/21#issuecomment-1701583527,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7CCKLBA6TK4FVXRFNCRGQTXYDMAXANCNFSM6AAAAAA4FEDTGU
.
You are receiving this because you commented.Message ID:
@.***>

as i expected the solution for fr is also not stable:
as the type: ORGANISATION contact: happens several times and the order the
whois server replies those contacts are not defined i now get markmonitor
i expect you need the data from nic handle holder-c:
GIHU100-FRNIC in this case
test2.py SIMPLISTIC: False
DEBUG: ### lookup: tldString: fr; dList: ['google', 'fr']
DEBUG: CACHE_STUB None
DEBUG: initializing default cache
DEBUG cache init SimpleCacheBase
init SimpleCacheWithFile
DEBUG: force: False
DEBUG cache get: google.fr
get: no data
DEBUG: timout: 30.0
DEBUG: [Querying whois.nic.fr]
[whois.nic.fr]
%%
%% This is the AFNIC Whois server.
%%
%% complete date format: YYYY-MM-DDThh:mm:ssZ
%%
%% Rights restricted by copyright.
%% See
https://www.afnic.fr/en/domain-names-and-support/everything-there-is-to-know-about-domain-names/find-a-domain-name-or-a-holder-using-whois/
%%
%%
domain:                        google.fr
status:                        ACTIVE
eppstatus:                     serverUpdateProhibited
eppstatus:                     serverTransferProhibited
eppstatus:                     serverDeleteProhibited
eppstatus:                     serverRecoverProhibited
hold:                          NO
holder-c:                      GIHU100-FRNIC
admin-c:                       GIHU101-FRNIC
tech-c:                        MI3669-FRNIC
registrar:                     MARKMONITOR Inc.
Expiry Date:                   2023-12-30T17:16:48Z
created:                       2000-07-26T22:00:00Z
last-update:                   2022-12-03T09:40:42.40624Z
source:                        FRNIC
nserver:                       ns1.google.com
nserver:                       ns2.google.com
nserver:                       ns3.google.com
nserver:                       ns4.google.com
source:                        FRNIC
registrar:                     MARKMONITOR Inc.
address:                       2150 S. Bonito Way, Suite 150
address:                       ID 83642 MERIDIAN
country:                       US
phone:                         +1.2083895740
fax-no:                        +1.2083895771
e-mail:                        @.***
website:                       http://www.markmonitor.com
anonymous:                     No
registered:                    2002-01-07T00:00:00Z
source:                        FRNIC
nic-hdl:                       MI3669-FRNIC
type:                          ORGANIZATION
contact:                       MarkMonitor Inc.
address:                       2150 S. Bonito Way, Suite 150
address:                       83642 Meridian
country:                       US
phone:                         +1.2083895740
fax-no:                        +1.2083895771
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
changed:                       2023-08-25T15:02:54.100903Z
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     associated
eppstatus:                     active
eligstatus:                    ok
eligsource:                    REGISTRAR
eligdate:                      2021-10-05T00:00:00Z
reachstatus:                   ok
reachmedia:                    email
reachsource:                   REGISTRAR
reachdate:                     2021-10-05T00:00:00Z
source:                        FRNIC
nic-hdl:                       GIHU101-FRNIC
type:                          ORGANIZATION
contact:                       Google Ireland Holdings Unlimited Company
address:                       70 Sir John Rogerson's Quay
address:                       2 Dublin
country:                       IE
phone:                         +353.14361000
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     associated
eppstatus:                     active
eligstatus:                    not identified
reachstatus:                   ok
reachmedia:                    email
reachsource:                   REGISTRAR
reachdate:                     2018-03-02T00:00:00Z
source:                        FRNIC
nic-hdl:                       GIHU100-FRNIC
type:                          ORGANIZATION
contact:                       Google Ireland Holdings Unlimited Company
address:                       Google Ireland Holdings Unlimited Company
address:                       70 Sir John Rogerson's Quay
address:                       2 Dublin
country:                       IE
phone:                         +353.14361000
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
changed:                       2022-10-15T05:41:14.918179Z
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     serverUpdateProhibited
eppstatus:                     associated
eligstatus:                    not identified
reachstatus:                   not identified
source:                        FRNIC

WHOIS request date: 2023-09-01T07:39:41.215977Z <<<

DEBUG: cache put: google.fr
DEBUG: Raw:  [Querying whois.nic.fr]
[whois.nic.fr]
%%
%% This is the AFNIC Whois server.
%%
%% complete date format: YYYY-MM-DDThh:mm:ssZ
%%
%% Rights restricted by copyright.
%% See
https://www.afnic.fr/en/domain-names-and-support/everything-there-is-to-know-about-domain-names/find-a-domain-name-or-a-holder-using-whois/
%%
%%
domain:                        google.fr
status:                        ACTIVE
eppstatus:                     serverUpdateProhibited
eppstatus:                     serverTransferProhibited
eppstatus:                     serverDeleteProhibited
eppstatus:                     serverRecoverProhibited
hold:                          NO
holder-c:                      GIHU100-FRNIC
admin-c:                       GIHU101-FRNIC
tech-c:                        MI3669-FRNIC
registrar:                     MARKMONITOR Inc.
Expiry Date:                   2023-12-30T17:16:48Z
created:                       2000-07-26T22:00:00Z
last-update:                   2022-12-03T09:40:42.40624Z
source:                        FRNIC
nserver:                       ns1.google.com
nserver:                       ns2.google.com
nserver:                       ns3.google.com
nserver:                       ns4.google.com
source:                        FRNIC
registrar:                     MARKMONITOR Inc.
address:                       2150 S. Bonito Way, Suite 150
address:                       ID 83642 MERIDIAN
country:                       US
phone:                         +1.2083895740
fax-no:                        +1.2083895771
e-mail:                        @.***
website:                       http://www.markmonitor.com
anonymous:                     No
registered:                    2002-01-07T00:00:00Z
source:                        FRNIC
nic-hdl:                       MI3669-FRNIC
type:                          ORGANIZATION
contact:                       MarkMonitor Inc.
address:                       2150 S. Bonito Way, Suite 150
address:                       83642 Meridian
country:                       US
phone:                         +1.2083895740
fax-no:                        +1.2083895771
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
changed:                       2023-08-25T15:02:54.100903Z
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     associated
eppstatus:                     active
eligstatus:                    ok
eligsource:                    REGISTRAR
eligdate:                      2021-10-05T00:00:00Z
reachstatus:                   ok
reachmedia:                    email
reachsource:                   REGISTRAR
reachdate:                     2021-10-05T00:00:00Z
source:                        FRNIC
nic-hdl:                       GIHU101-FRNIC
type:                          ORGANIZATION
contact:                       Google Ireland Holdings Unlimited Company
address:                       70 Sir John Rogerson's Quay
address:                       2 Dublin
country:                       IE
phone:                         +353.14361000
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     associated
eppstatus:                     active
eligstatus:                    not identified
reachstatus:                   ok
reachmedia:                    email
reachsource:                   REGISTRAR
reachdate:                     2018-03-02T00:00:00Z
source:                        FRNIC
nic-hdl:                       GIHU100-FRNIC
type:                          ORGANIZATION
contact:                       Google Ireland Holdings Unlimited Company
address:                       Google Ireland Holdings Unlimited Company
address:                       70 Sir John Rogerson's Quay
address:                       2 Dublin
country:                       IE
phone:                         +353.14361000
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
changed:                       2022-10-15T05:41:14.918179Z
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     serverUpdateProhibited
eppstatus:                     associated
eligstatus:                    not identified
reachstatus:                   not identified
source:                        FRNIC

WHOIS request date: 2023-09-01T07:39:41.215977Z <<<

domain_name, ['                       google.fr']
registrar, ['MARKMONITOR Inc.', 'MARKMONITOR Inc.', 'MARKMONITOR Inc.',
'MARKMONITOR Inc.', 'MARKMONITOR Inc.']
registrant, ['                      MarkMonitor Inc.', '
Google Ireland Holdings Unlimited Company', '
Google Ireland Holdings Unlimited Company']
registrant_country, ['                      US', '
US', '                      IE', '                      IE']
creation_date, ['                      2000-07-26T22:00:00Z']
expiration_date, ['                  2023-12-30T17:16:48Z']
updated_date, ['                  2022-12-03T09:40:42.40624Z']
name_servers, ['ns1.google.com', 'ns2.google.com', 'ns3.google.com', '
ns4.google.com']
status, ['                       ACTIVE', '
serverUpdateProhibited', '                    serverTransferProhibited', '
serverDeleteProhibited', '
serverRecoverProhibited', '                    associated', '
active', '                   ok', '                  ok', '
associated', '                    active', '
not identified', '                  ok', '
serverUpdateProhibited', '                    associated', '
not identified', '                  not identified']
emails, @.', @.', '
@.', @.']
registrant_organization, ['MarkMonitor Inc.', 'Google Ireland Holdings
Unlimited Company', 'Google Ireland Holdings Unlimited Company']
DEBUG: Clean: [Querying whois.nic.fr]
[whois.nic.fr]
%%
%% This is the AFNIC Whois server.
%%
%% complete date format: YYYY-MM-DDThh:mm:ssZ
%%
%% Rights restricted by copyright.
%% See
https://www.afnic.fr/en/domain-names-and-support/everything-there-is-to-know-about-domain-names/find-a-domain-name-or-a-holder-using-whois/
%%
%%
domain:                        google.fr
status:                        ACTIVE
eppstatus:                     serverUpdateProhibited
eppstatus:                     serverTransferProhibited
eppstatus:                     serverDeleteProhibited
eppstatus:                     serverRecoverProhibited
hold:                          NO
holder-c:                      GIHU100-FRNIC
admin-c:                       GIHU101-FRNIC
tech-c:                        MI3669-FRNIC
registrar:                     MARKMONITOR Inc.
Expiry Date:                   2023-12-30T17:16:48Z
created:                       2000-07-26T22:00:00Z
last-update:                   2022-12-03T09:40:42.40624Z
source:                        FRNIC
nserver:                       ns1.google.com
nserver:                       ns2.google.com
nserver:                       ns3.google.com
nserver:                       ns4.google.com
source:                        FRNIC
registrar:                     MARKMONITOR Inc.
address:                       2150 S. Bonito Way, Suite 150
address:                       ID 83642 MERIDIAN
country:                       US
phone:                         +1.2083895740
fax-no:                        +1.2083895771
e-mail:                        @.***
website:                       http://www.markmonitor.com
anonymous:                     No
registered:                    2002-01-07T00:00:00Z
source:                        FRNIC
nic-hdl:                       MI3669-FRNIC
type:                          ORGANIZATION
contact:                       MarkMonitor Inc.
address:                       2150 S. Bonito Way, Suite 150
address:                       83642 Meridian
country:                       US
phone:                         +1.2083895740
fax-no:                        +1.2083895771
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
changed:                       2023-08-25T15:02:54.100903Z
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     associated
eppstatus:                     active
eligstatus:                    ok
eligsource:                    REGISTRAR
eligdate:                      2021-10-05T00:00:00Z
reachstatus:                   ok
reachmedia:                    email
reachsource:                   REGISTRAR
reachdate:                     2021-10-05T00:00:00Z
source:                        FRNIC
nic-hdl:                       GIHU101-FRNIC
type:                          ORGANIZATION
contact:                       Google Ireland Holdings Unlimited Company
address:                       70 Sir John Rogerson's Quay
address:                       2 Dublin
country:                       IE
phone:                         +353.14361000
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     associated
eppstatus:                     active
eligstatus:                    not identified
reachstatus:                   ok
reachmedia:                    email
reachsource:                   REGISTRAR
reachdate:                     2018-03-02T00:00:00Z
source:                        FRNIC
nic-hdl:                       GIHU100-FRNIC
type:                          ORGANIZATION
contact:                       Google Ireland Holdings Unlimited Company
address:                       Google Ireland Holdings Unlimited Company
address:                       70 Sir John Rogerson's Quay
address:                       2 Dublin
country:                       IE
phone:                         +353.14361000
e-mail:                        @.***
registrar:                     MARKMONITOR Inc.
changed:                       2022-10-15T05:41:14.918179Z
anonymous:                     NO
obsoleted:                     NO
eppstatus:                     serverUpdateProhibited
eppstatus:                     associated
eligstatus:                    not identified
reachstatus:                   not identified
source:                        FRNIC

WHOIS request date: 2023-09-01T07:39:41.215977Z <<<

On Thu, Aug 31, 2023 at 8:48 PM Badreddine Lejmi @.***>
wrote:

I can handle this PR. It will not be so hard if I use the same technique
as I did for .fr
Regarding GDPR, it's the organization's name, not the person's name so not
it does not apply.
And even if it does apply, here the data processor is the user not the
tool in itself.
The only issue a user may have with this tool, and that already exists
actually, is the persistent file storage cache mechanism yet it's unrelated
to this specific issue.
—
Reply to this email directly, view it on GitHub
https://github.com/mboot-github/WhoisDomain/issues/21#issuecomment-1701583527,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7CCKLBA6TK4FVXRFNCRGQTXYDMAXANCNFSM6AAAAAA4FEDTGU
.
You are receiving this because you commented.Message ID:
@.***>

It's not stable but that's much better than it was. We can make it even better by comparing the registrant with the registrat and picking the first registrant that's different from the registrat (.lower()) if there is more than one registrant.

im currently thinking about adding a feature like
{
    "section: {
        "from_to": [
            r"<from regex",
            r"to regex"
        ],
        "extract": r"extract regex"
    }
}

to first extract a section (like one contact record group) and then extract
info from that section only for entries like whois google.sk
this possibly with a feature to extract based on info we extracted earlier
for contact sections that themselves do not provide what the contact is
about, like whois google.fr
On Fri, Sep 1, 2023 at 10:58 AM Badreddine Lejmi @.***>
wrote:

It's not stable but that's much better than it was. We can make it even
better by comparing the registrant with the registrat and picking the first
registrant that's different from the registrat (.lower()) if there is more
than one registrant.
—
Reply to this email directly, view it on GitHub
https://github.com/mboot-github/WhoisDomain/issues/21#issuecomment-1702409873,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7CCKLGD4D6N4SGZXE2M4KLXYGPTFANCNFSM6AAAAAA4FEDTGU
.
You are receiving this because you commented.Message ID:
@.***>

ok new code is now available in tld_regexpr that convers all existing re strings to functions.
see the def R(reString)
the function is called by whoisParser.py as func(textString) and should return a [str]
this means it is now easy to add more complicated parsing then one single regex
we can now make a func like:
def contextExtract(beginRe: str, endRe: str, findRe: str) -> Callable[[str], List[str]]:
      def f(textString: str) -> List[str]:
          extract_section = extract_from_to(beginRe, endRe)
          return  find_inSEction(extract_section, findReString)
    return f

to limit searched for something in a particular section of the whois cli result
the road is open for much more targeted searches
even based on information from other parts of the whois response

i updated the "sk" tld to use the new contextual extract

add findFromToAndLookForWithFindFirst contextual search based on a previous findFirst,
used in "fr" tld,
example google.fr,
{} is used to add to fromStr

how would you want to handle

whois pik.bzh -h whois.nic.bzh

Ok, we could progressively :

registrant by default is organization, second name (when redacted for privacy return the value anyway, yet we should also have a set of different redacted for privacy values and got an option to skip values in this case)
we'll had a tree-like structure like in this approach https://www.netmeister.org/blog/whois.html or whoisit one (IMHO, we should have both: a flat easy one and an automatically nested one in an advanced mode)

FYI, I've benchmarked whoisdomain, asyncwhois and whoisit against this registrant name issue and performance. It appears that whoisdomain is the fastest and close second in terms of quality. Test has to be reproduced in other machines because of network/caching issues.
❯ ./whoisdomain-benchmark.py
{'asyncwhois': {'count': 49,
                'duration':<PHONE_NUMBER>1399947,
                'end': 6710.12183199,
                'percentage': 26,
                'start': 6424.277741376},
 'whoisdomain': {'count': 44,
                 'duration':<PHONE_NUMBER>7400007,
                 'end': 6396.365353012,
                 'percentage': 24,
                 'start': 6200.824835038},
 'whoisit': {'count': 6,
             'duration': 27.91238160300054,
             'end': 6424.27773957,
             'percentage': 3,
             'start': 6396.365357967}}

added the proper parsing for google. from your test case https://github.com/mboot-github/WhoisDomain/issues/21#issue-1874402067, some actually have no organization or name (google.si, google.co.tz) some have no data or no registrant, a similar test for meta wold be nice if that is possible

What are meta words? Could you describe a little bit the test case you wish, I could write it.
Thanks for your quick PR, I thought I'll do it.
I think than in the long-term approach, an approched based on JSWhois approach is the most interesting: https://github.com/jschauma/jswhois/blob/main/src/jswhois.go / https://www.netmeister.org/blog/whois.html i.e:
(0. write pythonic code (adopt PEP8, black))

define a common nomenclature, flat AND tree based (for multiple entities) loosely inspired from RDAP but much simpler
categorize TLD by their NIC Whois format type
do some custom parsing capabilities for some TLDs that have a derived version of format
add RDAP support to this library for some TLD like .be who dropped Whois
add unit tests for every TLD - at least 3 different would be better
(Nice to have: 6. detect "privacy" comments)

JSwhois is certainly interesting to look at (item 2 could be derived from work at jswhois)

currently adding rdap would mean to introduce a additional dependency and parsing rdap is not that easy as they go endlessly on in lists ;-)
the option: withRedacted will include text that would normally be redacted

i finished all registrars i could find for the google test: see
cd /tests/
make google

meta is all domains owned by facebook
see: whois meta.com

Ok, for meta.com it was a bit hard so I tried with various facebook domains
It gaves me this result (quite good actually) on a old release, so it should be even better now:
✦ ❯ ./whoisdomain-benchmark.py facebook
Whois domain with parsers: ['whoisdomain', 'whoisit', 'asyncwhois', 'pythonwhoisalt'] on organization FACEBOOK for domains ['facebook.cn', 'facebook.com', 'facebook.fr', 'facebook.de', 'facebook.co.uk', 'facebook.net', 'facebook.gr', 'facebook.nl', 'facebook.se', 'facebook.fi', 'facebook.ae', 'facebook.cm', 'facebook.co', 'facebook.it', 'facebook.es', 'facebook.za', 'facebook.ca', 'facebook.pl', 'facebook.su', 'facebook.ru', 'facebook.tw', 'facebook.jp', 'facebook.au', 'facebook.nz', 'facebook.ar', 'facebook.mx', 'facebook.is', 'facebook.io']
{'asyncwhois': {'count': 3, 'duration': 9.292810824001208, 'percentage': '11%'},
 'pythonwhoisalt': {'count': 2,
                    'duration': 11.09157811201294,
                    'percentage': '7%'},
 'whoisdomain': {'count': 6,
                 'duration': 12.752617797988933,
                 'percentage': '21%'},
 'whoisit': {'count': 2, 'duration': 5.797948399995221, 'percentage': '7%'}}

thanks,  (.za has no whois server, nz has no registrar in the response)
i added a test for facebook that we can expand on
i will see if i can make a minimal rdap client we could use
and add some rdap info to the ZZ dict in tld_regexpr.py
currently adding public suffix list info (if library 'tld' is included on the running platform)
and im thinking on replacing all if self.pc.verbose: print stderr with: logging.debug(msg)

shall we close this one for now, we can add new issues for future work on  rdap , and json response with grouping


Precision of jnp.linalg.solve

Description
I noticed a difference in numerical precision between jnp.linalg.solve and np.linalg.solve.
Numpy:
import numpy as np
matrix = np.array([[ 1,                  0,  0],
                   [-1.+2.0958450e-06j,  1, -2.1878743-2.0878208e+06j],
                   [ 0,                  0,  1]], dtype=np.complex64)
vector = np.array([0, 0, 1], dtype=np.complex64)
np.linalg.solve(matrix, vector)

Output: array([0.             -0.j , 2.1878743+2087820.8j, 1.             +0.j ], dtype=complex64)

JAX:
from jax import numpy as jnp
matrix = jnp.array([[ 1,                  0,  0],
                    [-1.+2.0958450e-06j,  1, -2.1878743-2.0878208e+06j],
                    [ 0,                  0,  1]], dtype=jnp.complex64)
vector = jnp.array([0, 0, 1], dtype=jnp.complex64)
jnp.linalg.solve(matrix, vector)

Output: Array([-2.3841858e-07-4.9968840e-13j,  2.1878741e+00+2.0878208e+06j, 1.0000000e+00+0.0000000e+00j], dtype=complex64)

Switching to jnp.complex128 helps, but only reduces the error:
from jax import numpy as jnp
import jax
jax.config.update("jax_enable_x64", True)
matrix = jnp.array([[ 1,                  0,  0],
                    [-1.+2.0958450e-06j,  1, -2.1878743-2.0878208e+06j],
                    [ 0,                  0,  1]], dtype=jnp.complex128)
vector = jnp.array([0, 0, 1], dtype=jnp.complex128)
jnp.linalg.solve(matrix, vector)

Output: Array([8.8817842e-16+1.8614843e-21j, 2.1878743e+00+2.0878208e+06j, 1.0000000e+00+0.0000000e+00j], dtype=complex128)

I would have expected highest precision when switching both arrays to jnp.complex128, but if I only change the vector to jnp.complex128 and leave the matrix on jnp.complex64, the numerical error is gone:
from jax import numpy as jnp
import jax
jax.config.update("jax_enable_x64", True)
matrix = jnp.array([[ 1,                  0,  0],
                    [-1.+2.0958450e-06j,  1, -2.1878743-2.0878208e+06j],
                    [ 0,                  0,  1]], dtype=jnp.complex64)
vector = jnp.array([0, 0, 1], dtype=jnp.complex128)
jnp.linalg.solve(matrix, vector)

Output: Array([0.              -0.j  , 2.18787432+2087820.75j, 1.              +0.j  ], dtype=complex128)

When the matrix is jnp.complex128 and the vector jnp.complex64, there is still some error left.
Is there anything to learn here? Like a general rule of when to choose which combination of jnp.complex128 and jnp.complex64?
System info (python version, jaxlib version, accelerator, etc.)
jax:    0.4.26
jaxlib: 0.4.26
numpy:  1.26.4
python: 3.11.9 (main, Apr  6 2024, 17:59:24) [GCC 11.4.0]
jax.devices (1 total, 1 local): [cuda(id=0)]
process_count: 1
platform: uname_result(system='Linux', release='6.8.0-40-generic', version='#40~22.04.3-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 30 17:30:19 UTC 2', machine='x86_64')

$ nvidia-smi
Mon Aug 19 21:34:58 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1050 Ti     Off |   00000000:01:00.0 Off |                N[/]A |
| N/A   56C    P8           N[/]A / ERR!  |    3174MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      2630      G   [/usr/lib/xorg/Xorg]                            4MiB |
|    0   N/A  N/A     13661      C   venv/bin/python3.11                          3168MiB |
+-----------------------------------------------------------------------------------------+

JAX generally matches its internal computation to the precision of the inputs, so if you call a function with float32 inputs, it will perform the computation in float32. This makes sense because JAX is often used on accelerators like GPU and TPU, where hardware is not necessarily optimized for 64-bit computation.
By contrast, NumPy generally performs its computations in float64, regardless of the dtype of the inputs. This makes sense because NumPy only supports CPU backends, and modern CPU hardware mostly supports efficient 64-bit computations.
The net result is, if you want JAX to behave like NumPy, you need to (1) set jax_enable_x64 to True, and (2) make sure to cast your inputs to 64-bit (casting just one is often fine, becuase the function will promote inputs to a common type).
I think this explains everything you're seeing in your examples – please let me know if you still have questions!

Thank you for your answer! One doubt left are the casting mechanics behind my third and fourth example. With both inputs casted to 64-bit, the result seems to be less precise than the one from casting only the vector to 64-bit while leaving the matrix on 32.
So casting only one input and not both seems to improve the result. Is there a general rule or is it just a coincidence for this specific case?

I think the reason is that when you define your matrix as complex64 and then cast to complex128, you end up with a matrix containing rounded values, and so different numerics for the output is not unexpected:
M1 = jnp.array([[ 1,                  0,  0],
                [-1.+2.0958450e-06j,  1, -2.1878743-2.0878208e+06j],
                [ 0,                  0,  1]], dtype=jnp.complex64).astype(jnp.complex128)
M2 = jnp.array([[ 1,                  0,  0],
                [-1.+2.0958450e-06j,  1, -2.1878743-2.0878208e+06j],
                [ 0,                  0,  1]], dtype=jnp.complex128)

print(M1 == M2)
# [[ True  True  True]
#  [False  True False]
#  [ True  True  True]]


Clarify bug reporting process

I've been a little confused about where to file bugs, sometimes which gtk-rs repo to use and also when to file bugs upstream instead of in this repo. I want to make sure I'm being a good citizen and filing bugs ina  helpful matter and not just causing triage noise for this project. Therefore I think it'd be helpful if there was an official comment addressing these two things in the README for gtk. Something akin to:
Filing Bugs
Filing bugs upstream if possible (GNOME Bugzilla) is ideal if you believe the bug is specifically within the C libraries of GTK.
Otherwise if you believe the bug is within the Rust bindings, please file it in the appropriate repo within the gtk-rs family (glib, gio, gtk, etc.). If you are uncertain which repository to file the bug under, please file it under gtk.

Looking to close out some bugs and this seems like an easy one. Would this be suitable for merging, I can file a PR if so.

Seems like a good thing to have. :)


Faster and smoother dithered fading

The way we previously implemented dithered fading, the fading-in tile needed to be completely opaque before the fading-out tile could start fading. Otherwise, if both were dithered at the same time, you could see straight through both tiles for some fragments. @kring suggested that since the dither texture sample happens in screen-space, we could use the same fade percentage (dither threshold) for both fading-in and fading-out tiles, but flip the result of the dither sample for fading out tiles. The result is that we get a water-tight blending between the two LODs. This allows the transition to be smoother - since there are no longer two visibly distinct phases in the transitions, and faster - since the old tile doesn't need to wait for the new tile to fully fade in.

Thanks for the pull request @nithinp7!

:heavy_check_mark: Signed CLA found.

Reviewers, don't forget to make sure that:

[ ] Cesium for Unreal Samples works.

Thanks @nithinp7! Please add this to the changelog in main.


How to unit test with AutocompleteDropdown using Jest?

I have a screen that imports AutocompleteDropdown and adds it as a component.
I am trying to unit test this screen using the react-native testing library.
When I render the screen in the test, I get the following error:
    Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but
got: undefined.  You likely forgot to export your component from the file it's defined in, or you might have mixed up default
 and named imports.

    Check the render method of `AddHabitScreen`.
{
      21 |
    > 22 |     render(
         |           ^
      23 |         <NavigationContainer>
      24 |             <AddHabitScreen />
      25 |         </NavigationContainer>

The AddHabitScreen is the screen that has the import statement as follows:
# AddHabitScreen.js
import { AutocompleteDropdown } from 'react-native-autocomplete-dropdown';

const AddHabitScreen = ({ navigation, route }) => {
...
<AutocompleteDropdown
...
/>
...
}

export default AddHabitScreen;

This code works when I test the interactions manually on a device but fails during the test. I am assuming mocking is the way to prevent this, but I haven't found a way to properly mock this library.
Any ideas on how I could mock this to avoid this error?

One more observation.
I add a mock file as follows in the __mocks__ directory.
# react-native-autocomplete-dropdown.js

import React from 'react';
import { View } from 'react-native';

const AutocompleteDropdown = (props) => <View {...props} />;

export default AutocompleteDropdown;

With this, I get the same render error as before but along with that, I get an additional error specifically showing AutocompleteDropdown as follows:
console.error
  Warning: React.jsx: type is invalid -- expected a string (for built-in components) or a
  class/function (for composite components) but got: undefined. You likely forgot to export your
  component from the file it's defined in, or you might have mixed up default and named imports.

  Check the render method of `AddHabitScreen`.
      at navigation (<path>\src\habits\AddHabitScreen.js:47:27)
      at children (<path>\node_modules\@react-navigation\core\src\EnsureSingleNavigator.tsx:20:49)
      at initialState (<path>\node_modules\@react-navigation\core\src\BaseNavigationContainer.tsx:82:7)
      at value (<path>\node_modules\@react-navigation\native\src\theming\ThemeProvider.tsx:11:41)
      at theme (<path>\node_modules\@react-navigation\native\src\NavigationContainer.tsx:55:5)

    83 |                         }}
    84 |                     >
  > 85 |                         <AutocompleteDropdown
        |                         ^
    86 |                             clearOnFocus={false}
    87 |                             closeOnBlur={true}
    88 |                             closeOnSubmit={false}

Not sure if that helps in any way to debug this issue.


Update dependencies and UI5 version

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @akudev ,
please also do a quick re-test whether all works fine. I upgraded the dependencies with:
yarn upgrade-interactive --latest

since this also upgrades the 0.x version which somehow are handled als major version jumps for the >=1.x.x versions. This is the regular behavior of semver that minor versions for 0.x.x releases are handled like this.
Thanks...
Peter


import cve's automatically with a queue where CVe's can be reviewed

It would be nice to import CVE's automatically using an API maybe such as https://cve.circl.lu/api/ or https://nvd.nist.gov/vuln/data-feeds and add a review / queue to the security tracker to review these issues.
An idea is to create a new view /todo/cve and show a list of CVE's who don't have at least two signoffs and show the following information:
CVE-XXX-XX-XX (a link) product and version, description and links. If the product can be found in our Package model show the package and version or do a fuzzy match on our Packages and show a list? There should be a button to create an AVG with this CVE pre-filled in.
The CVE signoffs should be a table cve_signoff that stores the CVE id, team member, signoff time
Open questions:

How do we map the CVE type from the CVE feed to our CVE type?
We lack the package name and package version in the CVE model which we could extract from the CVE feed?

@ysf can you take a look at this?

It would be great to have a sanitized dump to get a feel of how the data has to look later on and generally work with the tracker.

Hi @ysf, sad that we didn't met on 36C3 :)
You can find dumps of our data via putting .json in your URL bar.
For example for a CVE:
https://security.archlinux.org/CVE-2019-6474.json
For an AVG:
https://security.archlinux.org/AVG-1032.json
and so on. Hope this helps.

Yeah, but I don't want to scrape those and add them under an anonymous user into my local database if those could be easily sanitized to get a realistic working environment. Actually it's a time issue, if it's quicker to scrape and import than to sanitize and share I'll gladly do it.

I think there should be two views on the tracker for this data.

One view that parses the recent/modified list and contains the following data

CVE
pkgname
version
severity

Obviously the entries needs to be as few false positives as possible. We should also enable easy creation of an AVG group and do some basic matching on pkgname and version to find existing AVGs.

One view to search CVEs which are reserved, and not public, so we can have a todolist when details are filled from the recently modified list


#46 Error in readme file

In the following snippet
public class GenreViewHolder extends GroupViewHolder
  private TextView genreTitle;
  public GenreViewHolder(View itemView) {
    super(itemView);
    genreTitle = itemView.findViewById(R.id.genre_title);
  }
  public void setGenreTitle(ExpandableGroup group) {
    genreTitle.setText(group.getTitle());
  }
}
There is an error at
public class GenreViewHolder extends GroupViewHolder **{**
the opening bracket is missing.

There are lots of error please re-check the readme file once


Pre-Training Code of MarkupLM

I couldn't find the pre-training code of MarkupLM. Is it possible to share the same? @lockon-n

@TejasDuseja We do not have plans to release the pre-training code yet.


Chore/job scheduling

closes #543
implementing using Quartz.net

ready to review. I've created UpdateProjectMetadataJob as an example of how we might use this. That said, I'm not sure it's the best example for this. The whole point of this was to shift work being done during a web request to be done in another thread so the request can complete before the task is done. However in this case the work required to record the task and queue it is just about as much work as actually doing the task (updating metadata), so I don't think this actually makes sense here, at least not as a persistent job.
I'm going to look into setup up a second scheduler for in memory only jobs that we can afford to lose in case of a server restart, but will be faster to queue the job.

@myieye sadly you can only add an activity link when creating it, this is an API limitation from dotnet API.
Thanks for the review. I'm going to merge this in.


21-add-option-to-revert-to-default-storage-state-on-serializer-parsing-error

This PR was included in release version 1.0.0-beta.8 :tada:

This PR was included in release version 1.0.0 :tada:


Use observers to trigger learners

At EOD 5:01, maybe it's worthwhile triggering all the learners by registering them as observers to the main thread that run.

Implemented spawning new threads. Will not do observers.


Problem when run ionic run ios mac os

Caught exception:
Error: Command failed: cordova plugin add --save cordova-plugin-device
(node:839) fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.
(node:839) fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.
info trying registry request attempt 1 at 09:44:35
http GET http://registry.cordova.io/cordova-plugin-device
info retry will retry, error on last attempt: Error: connect ETIMEDOUT <IP_ADDRESS>:80
info trying registry request attempt 2 at 09:46:01
http GET http://registry.cordova.io/cordova-plugin-device
info retry will retry, error on last attempt: Error: connect ETIMEDOUT <IP_ADDRESS>:80
info trying registry request attempt 3 at 09:48:18
http GET http://registry.cordova.io/cordova-plugin-device
Error: connect ETIMEDOUT <IP_ADDRESS>:80
at Object.exports._errnoException (util.js:1007:11)
at exports._exceptionWithHostPort (util.js:1030:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1080:14)
at ChildProcess.exithandler (child_process.js:207:12)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:852:16)
at Socket.<anonymous> (internal/child_process.js:323:11)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at Pipe._handle.close [as _onclose] (net.js:492:12)

ionic info :
Your system information:
Cordova CLI: 4.2.0
Ionic Framework Version: 2.0.0-beta.6
Ionic CLI Version: 2.0.0-beta.32
Ionic App Lib Version: 2.0.0-beta.18
ios-deploy version: Not installed
ios-sim version: 5.0.8
OS: Mac OS X El Capitan
Node Version: v6.2.2
Xcode version: Xcode 7.3.1 Build version 7D1014

Hello! Thanks for opening an issue with us. It seems that you are running an older version of the cordova cli. Would you mind running sudo npm install -g cordova and trying again please? Thanks for using Ionic!

Hello all! As it seems it has been a while since there was any activity on this issue i will be closing it for now. Feel free to comment if you are still running into this issue. Thanks for using Ionic!


Multiple QR Code at once

I need to scan multiple QR codes at once in one scan. Is it possible for this library? Can you please help me with this?

hey, @rutvikrockers, what do you mean with multiples QR codes to scan at once? can you provide us an example of how the QR codes should be represented to scanning?
however, just to let you know in advance, there's no way currently, to scan more than just a single QR code because the scanner just looks on the center of the camera to match a QR code. So, regardless, it will be required a new feature to be built to make this possible.
but I'm pretty confused about how should be this feature because I couldn't even imagine that it would be possible.

because the scanner just looks on the center of the camera to match a QR code

It's really doesn't. onRead just passes data from RNCamera's onBarCodeRead callback which scans barcodes... in a very strange order. I did some tests. I printed 3 qr codes on a piece of paper with different data ("25", "42" and "75") near each other.
First, just a regular scan. It picks up the one on the top ("25"). It ignores the mask which, after looking at the code, is just there for decoration.

Secondly, I covered up the first code. It immiediately picked up the second one with "42" in it.

Lastly i covered up both codes. It then scanned the last code with "75". Inb4 it's not about the size - it's not like it scanned the one on the top the fastest because its the biggest. On the other side I printed codes with the same size and did exactly the same test and the results didn't vary at all.

The solution MIGHT be to use onGoogleVisionBarcodesDetected instead of onBarCodeRead which outputs an array of  barcodes it scanned. Then we would have to somehow filter barcodes that are outside the mask. But then there's a problem: I found that onGoogleVisionBarcodesDetected isn't as reliable as the regular onBarCodeRead - it sometimes looses some barcodes, sometimes its bounds are way off etc.
I studied various react-native scanners for a few days now and im sick of it lol
I only need a scanner that scans reliably 1cm*1cm qr codes but only inside the mask. Is that really too much to ask for? 😄

It looks like it was to much to ask :)
Looking for a similar feature as we have parcels which containing multiple barcodes / qr codes and I like to be able to scan them all and pick the one that fits us


Re-implement PDFHistory

Please refer to the individual commit messages.
@timvandermeij I know that this is a large PR, but would you be willing to try and review it?
I'd suggest that you completely ignore the combined diff, and instead look at each patch on its own.
Also, we should make sure this is landed at beginning of a development-cycle (see future dates), to allow more time for follow-ups.

Two things I'm wondering:

Does this also address the concerns in #6200, as that PR also seems to work on the PDF history code?
Maybe this is a long shot, but would it be possible to write unit tests for this PR? We do not need complete coverage, but some unit tests on the basic functionality would be nice to have. What I was thinking about is opening a PDF, changing the page programmatically and checking the history state to make sure that it is updated. Do you think that that would be possible, or would that be very hard and should we keep that for later?

Does this also address the concerns in #6200, as that PR also works on the PDF history code?

Yes, I believe that it does (it works in my testing).

Maybe this is a long shot, but would it be possible to write unit tests for this PR?

I though about that, but it seemed quite difficult to add meaningful tests without a UI testing framework in place.

@Rob--W Could you check if this PR solves the issue you adress with #6200 ? If so, we can probably close that PR and continue here as this is a more complete rewrite of the history code.
@Snuffleupagus Good, let's keep that for later then. I had hoped it would be easy, but indeed it can be hard without such a framework.

@timvandermeij I haven't tested this code yet, but I looked through it.
The current history logic (insofar relevant for #6200) is 1) going back in history, 2) rewriting the history state and 3) going forward in history. Step 1, 2 and 3 are currently exected right after each other, while in Chrome it is necessary to wait a little bit between each step. This was fixed by #6200.
I could be mistaken, but at the first sight it seems that this PR does not use navigate-back-rewrite-and-navigate-forwards logic, and instead just keeps replacing / pushing new items in the history. If that is true, and this PR gets merged, then #6200 becomes obsolete.
@Snuffleupagus Could you confirm whether my assessment is correct?

What's the status of this PR? https://github.com/mozilla/pdf.js/pull/6200#issuecomment-121197353

@Snuffleupagus
Actually, if you're willing to live with this issue for just a little while longer, it might actually be fixed "for free" by the PDFHistory rewrite that I'm working on.

This "a little while longer" is already taking over 2 months, and I'd like to publish the Chrome extension with a fix to the problem. If you expect this to take much more time, could #6200 be merged in the meanwhole, to improve the usability of the location bar in Chrome/Opera/Safari?

@Rob--W Apparently I forgot to answer your question, my apologies!

[From https://github.com/mozilla/pdf.js/pull/6272#issuecomment-124991481] Could you confirm whether my assessment is correct?

Yes, that would be correct!
The current code is not actually working as intended, and the implementation is also fairly complicated. Since changing the hash of a loaded document could be considered somewhat of an edge-case, this PR simplifies the implementation to not try to go back (and forward) and modify the previous history entry when the hash is changed by the user.

This "a little while longer" is already taking over 2 months, [...]

I agree that is unfortunate; but given that this is a pretty large PR, I don't think @timvandermeij has had time to review it yet.

If you expect this to take much more time, could #6200 be merged in the meanwhole, to improve the usability of the location bar in Chrome/Opera/Safari?

In my testing, PR #6200 unfortunately seem to introduce other issues; hence I'm not willing to merge it into master in its current form. (And I'd rather not spend a whole lot more time testing it, since I still hope that we'll get rid of the code in question.)

In my testing, PR #6200 unfortunately seem to introduce other issues; hence I'm not willing to merge it into master in its current form. (And I'd rather not spend a whole lot more time testing it, since I still hope that we'll get rid of the code in question.)

I'm not aware of any regressions in #6200. What are those?

I currently do not have any time to properly review this patch, unfortunately. Perhaps @brendandahl can review this?

@Snuffleupagus Could you rebase this patch? Unfortunately I'm still lacking enough time to properly review this, but given the current wave of reviews, probably either Yury or Brendan can take a look at this. I think the patch is too good to leave it laying around!

@yurydelendik Do you have time to review this?

@Snuffleupagus Is there any way in which we can split this PR into multiple smaller PRs? I'm willing to review this, but it would be much easier (and less error-prone) if it were split in smaller chunks? For example create a PR for the second commit about getDestinationHash (as it seems that that is rather independent of the PDFHistory rewrite) and instead of ripping everything out and rewriting it all, incrementally improve the current situation, for example by first creating a PR to streamline the event listener binding from https://github.com/Snuffleupagus/pdf.js/commit/8704aa60fc4ef177bca2bbbfdd6315c10f108804#diff-3b4ebbe223f93c9123a754b4845844e7R456.
It could be somewhat similar to what I did for the initial annotation layer rewrite. I tried to do everything at once, but soon came to the conclusion that is was easier and less error-prone when doing it in smaller parts. It will help to get this functionality landed sooner. What do you think?

Is there any way in which we can split this PR into multiple smaller PRs?

I really tried that when writing the code, but in the end I didn't do so because of the very high risk of introducing severely broken intermediate states. (Note that the current implementation suffers from many issues.)

For example create a PR for the second commit about getDestinationHash (as it seems that that is rather independent of the PDFHistory rewrite)

I'm not totally confident that that wouldn't break the current PDFhistory (even more than it currently is), but I'll try to look into that since it should be an easy win if it's possible.

It will help to get this functionality landed sooner. What do you think?

I totally understand this, but I just don't know how I'd be able to re-factor PDFHistory in steps :-(
The new implementation makes fever assumptions about the state, in order to prevent many of the issues with the current code, so I'm simply not sure if/how this would work.
I'll make another attempt to see if this can be split into multiple PRs, but I cannot promise that it will work!

Fortunately PR #7341 has been merged, so if we can rebase this PR onto the current master it should be more manageable in terms of size and complexity.

Note that I closed this PR, since it's very badly bit-rotten by now. I'm currently testing https://github.com/mozilla/pdf.js/compare/master...Snuffleupagus:rewrite-pdf-history, and if all goes will submit that soon.


curl -XPUT syntax error

used the following with a single quote at the end and it worked
'{
"mappings": {
"logs" : {
"properties" : {
"timestamp": {"type": "date"},
"geo": {"type": "geo_point"},
"city": {
"type": "string",
"index": "not_analyzed"}
}
}
}
}'

Fixed by ac4f05148b75acd610792aaa7f5697341613aeeb


Use of inline styles prevents good security via CSP

I'm working on a site that uses this plugin (v0.1.19). To improve site security, I'm adding a  Content Security Policy which, among other things, does not allow inline styles. The security risk posed by these is covered in this talk. In general, it's stuff like "if you allow inline styles, an attacker might somehow get <style> tags into your site that add deceptive elements, query for information in the page using attribute selectors, send data back to the attacker by setting specially-named background image URLs, etc."
We want to make that impossible. As an example CSP, if I send the very permissive header Content-Security-Policy:default-src *, it means "you can source scripts or styles or anything else from anywhere". But because that header doesn't whitelist inline scripts or styles, it implicitly forbids them.
When I load a page in my app that uses this plugin and sends that CSP header, I get an error in the Chrome developer console:

Refused to apply inline style because it violates the following Content Security Policy directive: "default-src *". Either the 'unsafe-inline' keyword, a hash ('sha256-7BWCoAyyyh4N57ifiNqsR2logKoVQnqkizK8Ya5j02g='), or a nonce ('nonce-...') is required to enable inline execution. Note also that 'style-src' was not explicitly set, so 'default-src' is used as a fallback.

Clicking on the error message gives me a stack trace that isn't super clear, but I found that if I comment out these lines that create and append a <style> tag, the warning disappears.
Could you instead provide this style as sticky-table-headers-print-styles.css or something similar to allow setting a CSP?

Another possible reason to change this - it appears that the current code is appending that <style> tag multiple times, which is unnecessary, right?

For those interested this works better and does not even use js. It is a bit more work to implement.
https://codepen.io/tfzvang/pen/WQBwVo

I am also running into an issue where CSP is flagging this script. It looks like it's still there in latest. Has there been any plans to update and improve it?


Get clippy CI job working

See the individual commits for details. Roughly:

Stop using deprecated features of chrono. This requires bumping the minimum chrono version.
Address all Clippy warnings.
Get the MSRV job in CI to pass, by raising the MSRV to 1.63.0.

@erickt PTAL at this. This is needed for any PR to pass the GitHub Actions CI. In particular, this is blocking #400 which updates to ring 0.17.


Retain significant whitespace

Happy for you to use or change or drop this like a bad habit. Maybe an argument to the *_string and *_stream functions to retain whitespace would be better.
No worries and thanks for your library!

Uh not sure where those other commits came from


Errors

Has anyone encountred these 2 errors:
Ollama server not found, please ensure the ollama Docker container is running! If so you may have to take down the docker compose and put it up again
Milvus database not found, please ensure the milvus-standalone, etcd and minio Docker containers are running! If so you may have to take down the docker compose and put it up again

Assuming you're going for an GPU installation, ensure your CUDA drivers are at the version required to run Concierge. That said, the errors from your terminal would be more helpful for solving your issue. Assuming the issue is with your CUDA drivers as it was with mine, try the following:
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt install nvidia-driver-510
sudo reboot

Assuming you're going for an GPU installation, ensure your CUDA drivers are at the version required to run Concierge. That said, the errors from your terminal would be more helpful for solving your issue. Assuming the issue is with your CUDA drivers as it was with mine, try the following:
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt install nvidia-driver-510
sudo reboot

Hello,
Thank you for the reply, here are the errors:
Concierge
Collection Management
Loader
Prompter
Loading Language Model: 3.83GB [11:56, 5.74MB/s]
MilvusException: <MilvusException: (code=2, message=Fail connecting to server on <IP_ADDRESS>:19530. Timeout)>
Traceback:
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 584, in _run_script
exec(code, module.dict)
File "/home/sam/Downloads/concierge-main/pages/Loader.py", line 21, in
ensure_collections()
File "/home/sam/Downloads/concierge-main/concierge_streamlit_lib/collections.py", line 17, in ensure_collections
st.session_state[COLLECTIONS] = get_collections_cached()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 168, in wrapper
return cached_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 197, in call
return self._get_or_create_cached_value(args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 224, in _get_or_create_cached_value
return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 280, in _handle_cache_miss
computed_value = self._info.func(*func_args, **func_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sam/Downloads/concierge-main/concierge_streamlit_lib/collections.py", line 13, in get_collections_cached
return get_collections()
^^^^^^^^^^^^^^^^^
File "/home/sam/Downloads/concierge-main/concierge_backend_lib/collections.py", line 32, in get_collections
connect()
File "/home/sam/Downloads/concierge-main/concierge_backend_lib/collections.py", line 4, in connect
connections.connect("default", host="<IP_ADDRESS>", port=19530)
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/pymilvus/orm/connections.py", line 414, in connect
connect_milvus(**kwargs, user=user, password=password, token=token, db_name=db_name)
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/pymilvus/orm/connections.py", line 365, in connect_milvus
gh._wait_for_channel_ready(timeout=timeout)
File "/home/sam/Downloads/concierge-main/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 146, in _wait_for_channel_ready
raise MilvusException(

Is there a way to convert to a CPU install?

For a CPU install, I assume your best bet will be to choose the default CPU option when running install.py.

Any way you can explain whats causing the above errors?

Since you're interested in leveraging AI tools, I recommend showing your errors to your favorite AI tool for an explanation. Here's what ChatGPT 4 says about it:
The error you encountered, MilvusException: (code=2, message=Fail connecting to server on <IP_ADDRESS>:19530. Timeout), suggests that the Python script is unable to establish a connection to the Milvus server, which is expected to be running on the local machine at port 19530. This could be due to several reasons:
Milvus Server Not Running: Ensure the Milvus server is up and running on your local machine. You may need to start the server if it isn't running.
Incorrect Port: Double-check that the Milvus server is configured to listen on port 19530. This configuration can typically be found in the Milvus server's settings or configuration files.
Firewall or Network Issue: Verify that your firewall settings or network configuration are not blocking connections to this port.
Server Overload or Timeout Settings: If the server is running but overloaded, it might not respond quickly enough, leading to a timeout. You may consider increasing the timeout setting in your script if possible, or ensure the server has sufficient resources and isn't overloaded.
To troubleshoot and resolve this issue, you could take the following steps:
Check Server Status: Use tools like systemctl status (for Linux) or relevant commands in other operating systems to check if the Milvus server is actively running.
Review Logs: Check the logs of the Milvus server for any errors or messages that could give more insight into why the connection is failing.
Increase Timeout: Modify the timeout setting in the connection parameters to see if giving more time for connection helps. This is typically an argument you can pass to the connect() function in the Milvus client.
Try Manual Connection: Try to manually connect to the server using the same address and port with a simple script or tool to verify that the server is accessible.

Whats the command to remove/uninstall so I can reinstall it?

Running install.py will allow you to remove previously installed components.

0.3.0 release switches from Milvus to OpenSearch database backend, please create a new issue if you're still having trouble with 0.3.0.


Immediate Attention Required

I have Posted private video on Youtube please check with same https://youtu.be/l8GwBvgFwP4

I Checked  the code and found that  feeType in response data is now fee_type

@siddhesh1770 it was always fee_type.


[BEAM-12007] Set message ordering key for Google Pubsub

Implemented ordering key for Google Pubsub

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[x] Update CHANGES.md with noteworthy changes.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @chamikaramj @charlesccychen

Beam PCollections do not guarantee ordering of elements

I think what would be useful to know from the author, is what they would expect the result to be if they set this key in beam.
@victuos, were you expecting the result of this to be message ordering preserved end to end through your pipeline? If so, that may not be the result here.

I think what would be useful to know from the author, is what they would expect the result to be if they set this key in beam.

We would expect that when a consumer of a subscription, with ordering enabled, would receive the outputted messages (with the same ordering key) in the order they were published. The order in which these messages are published is as far as I know determined by the timestamp of each element.
Being able to set the ordering key will not ensure end to end in order message processing. But if we group elements in a window by the ordering key, and output those elements with a specific timestamp, we can ensure they're published in order (for that window) and ordering can be enabled on subscriptions.

We would expect that when a consumer of a subscription, with ordering enabled, would receive the outputted messages (with the same ordering key) in the order they were published.

This will likely not be achieved by this PR. The beam model does not preserve ordering within a PCollection, allowing elements to be reordered arbitrarily.

The order in which these messages are published is as far as I know determined by the timestamp of each element.

I do not believe a Beam runner is required to process windows in timestamp order: if processing of a given window stalls, another window can begin to proceed concurrently. In this case, it would result in disordered messages between windows.

@victuos @dpcollins-google - What would be the next step on this PR?

Thank you @victuos for doing this. Unfortunately, I don't believe this will work as written since Beam will not preserve any form of order on the publish side through to Pub/Sub for the ordering key to be meaningful. The Beam model is complex and not all aspects are well documented, this might be one of those areas to improve. I would suggest closing this for now and keeping this jira open for a future complete implementation. @victuos, what do you think?

@victuos @dpcollins-google - What would be the next step on this PR?

Thank you @victuos for doing this. Unfortunately, I don't believe this will work as written since Beam will not preserve any form of order on the publish side through to Pub/Sub for the ordering key to be meaningful. The Beam model is complex and not all aspects are well documented, this might be one of those areas to improve. I would suggest closing this for now and keeping this jira open for a future complete implementation. @victuos, what do you think?

Unfortunately I agree, so I'll close the PR.
We have set up an internal meeting with Google devs on how we should continue. I will update the Jira ticket once I have more information.


Add message on /assign and bidding message Hello! You've been assigned to repo issue #123. You have 24 hours to complete a pull request. To place a bid, type /bid [amount in BCH]. Thank you!

-[ ] incorporate bidding functionality to integrate with BLT Bid on Bugs

@DonnieBLT can you assign me this issue ?

assign this to me


Contextual Information in Error Messages

Is your feature request related to a problem? Please describe.
When parsing programming languages and other grammars with nested structure, it is possible that layers of context are passed through. One example is the filename itself, which is already incorporated into error messages when using parseFromFile. Another example would be reporting syntax errors within functions.
Describe the solution you'd like
New combinators can be added to allow for the tracking of contextual information during the parse, but this also requires the formatting of contextual information to be added to ErrorBuilder: since these methods are abstract, this requires a breaking change to the API.

I think that we can actually add this via a non-breaking case. The idea is to have a subclass ErrorBuilder1 or something, that acts like the old Revision mixins but in reverse, where you have to opt-in to new features, not out of them. Then the internal formatting logic must have something like:
errBuilder match {
  case builder: ErrorBuilder1[Err] => /* new logic */
  case builder: ErrorBuilder[Err]   => /* old logic */
}

This will mean it can handle all the new functionality and fall back to old ones if the user implemented an old type.


Double Decoding of the QueryParams

What happened:
The code double decodes the query params resulting in malformed queryParam or altered queryParam.
What did you expect to have happen:
The queryParam to be return as such it was called.
How to reproduce this (as precisely and succinctly as possible):
Simply use the library and call with encoded queryParam and see the double decoding happening.
Anything else you think we should know?
The fix is in https://github.com/awslabs/cognito-at-edge/blob/main/index.js#L180
Removing the extra decoding might solve the issue.
Environment:

version of cognito-at-edge being used: 1.1.0
node version of code base which uses cognito-at-edge: 14x

Hi @AbhiPartha ,
Thanks for opening this issue and submitting a PR. I'm currently testing it and I'm observing some unexpected encoding differences between my test setup and another environment which uses the package. I'll keep you updated.

Merged and will be included in the next release.


member.getRoles() on MessageReactionRemoveEvent returns NullPointerException

General Troubleshooting

[Y] I have checked for similar issues.
[Y] I have updated to the latest JDA version.
[Y] I have checked the branches or the maintainers' PRs for upcoming bug fixes.

Bug Report
Using method member.getRoles() on MessageReactionRemoveEvent returns NullPointerException
Expected Behavior
Should return a list of users roles in List format
Code Example or Reproduction Steps
	@Override
	public void onMessageReactionRemove(MessageReactionRemoveEvent e) {
		System.out.print(e.getMember().getRoles());
	}

-->
Code for JDABuilder or DefaultShardManagerBuilder Used
DefaultShardManagerBuilder builder = DefaultShardManagerBuilder.createDefault(token);
builder.addEventListeners(new Class());
ShardManager sm = builder.build();
Exception or Error
java.lang.NullPointerException

That event won't provide the member, only the member's id. I guess you can figure the solution out from that.

@diniamo The event provides the member but it can be null. See the docs.
You could retrieve the member by its id tho. If you just want to remove a role, the method to do so also takes the member id so there would be no need to get the member first. Also you don't need to check if the member actually has the role before removing it, see here.

This is intended behavior since the cache doesn't know about the member here. Use retrieveMember instead.


Why charts don't support copy functionality?

Currently, we can only create a chart from scratch, instead of modifying it based on an existing chart。

Currently, we can only create a chart from scratch, instead of modifying it based on an existing chart。

I don't understand this issue, Could you detail it?

Currently, we can only create a chart from scratch, instead of modifying it based on an existing chart。

I don't understand this issue, Could you detail it?

Similar to copy action in role. As shown below

This way I can quickly create a new chart based on an existing chart。

Hey @LuPan2015 you're correct that Duplicating / Copying charts doesn't exist at the moment as a feature.
Two workarounds:

Export and Import the chart and give it a new name.
OR

Open the chart in Explore and when you click Save, save it as a new chart with a new name (don't overwrite).


Merging Delay Scheduling to Master

Naive implementation of the delay scheduling algorithm.

Hi @MikeSchouw, I see you guys made some good progress. Is there any reason why you closed the pull request? Anything I can help with perhaps?

Hi @fabianishere, the pull request I created was meant for our fork, but I made a small mistake. We are still testing everything to make sure we deliver quality solutions! Thank you for the attention :)

Ah I see. Looking forwards to your contributions!


RUM Action Dropped, because another action is still active for the same view

Thanks for taking the time for reporting an issue!
Describe what happened
Console - datadog showing RUM Action (TAP on ) was dropped, because another action is still active for the same view
Steps to reproduce the issue:
Fragment 1 -> Fragement 2 / Activity -> press back -> and it show the Warning of RUM action dropped
Note :- though it shows warning still the Action is recorded in RUM.
One more way to reproduce -
GlobalRum.get().startUserAction(RumActionType.CUSTOM, eventName, extraData)
GlobalRum.get().addUserAction(RumActionType.CUSTOM, eventName, extraData)
GlobalRum.get().stopUserAction(RumActionType.CUSTOM, eventName, extraData)
The above will show warning
RUM Action (CUSTOM on "eventName") was dropped, because another action is still active for the same view
Describe what you expected:
How to fix / avoid Action drop warning.
Additional context

Android OS version: - 12
Device Model: - Checked on Emulator and One Plus as well
Datadog SDK version: - com.datadoghq:dd-sdk-android:1.13.0
Versions of any other relevant dependencies (OkHttp, …): retrofit - 2.9.0 , okhttp3.okhttp.okhttpclient- 3.14.9
Proguard configuration: not enabled
Gradle Plugins: Gradle - 4.1.3

Code - for initialising - Datadog
val configuration = Configuration.Builder(
           rumEnabled = true,
           crashReportsEnabled = true,
           logsEnabled = true,
           tracesEnabled = true
       ).trackInteractions()
           .useSite(DatadogSite.US1)
           .trackBackgroundRumEvents(true) // may impact billing as additional sessions created
           .setUploadFrequency(UploadFrequency.AVERAGE)
           .setUseDeveloperModeWhenDebuggable(BuildConfig.DEBUG)
           .useViewTrackingStrategy(MixedViewTrackingStrategy(trackExtras = true))
           .setBatchSize(BatchSize.MEDIUM)
           .build()

       val credentials = Credentials(clientToken, environmentName, appVariantName, applicationId)
       Datadog.initialize(this, credentials, configuration, TrackingConsent.GRANTED)
       GlobalRum.registerIfAbsent(RumMonitor.Builder().build())

Hi @rupakTj , you're mentioning that this happen while also navigating and pressing the back button. With just the three lines you provided, we were unable to reproduce the issue, could you please provide more context around your example?

Closing this ticket due to inactivity, feel free to re-open it if you still experience this issue.


cc2file - Medium update

Please describe the changes this PR makes and why it should be merged:
This update brings some features:

Ability to limit execution to 5 CCs
Ability to show/hide the info of each CC, in message's content and/or attachment's content (it's inserted as a comment)²
Ability to blacklist CCs to maintain its code private¹

And some minor changes:

sendLongMessage template now uses an sdict rather than a cslice
Heavily-commented code: This is mostly for people who has to troubleshoot my code, if there are problems in a future

Status
I didn't really test every single combination of options, but yeah, it works ;)
Some improvements that can be done:

Allow some users/roles to override blacklist?
Parse $firstLine's content, extract CC info and print it in a format similar to this repo's Leading comments. Something like:

//Before:
{{/* CC INFO: #20 - Contains: `viva` - Case sensitive trigger: `false` - Group: `Respuestas` */}}

//After:
{{/*
	Description:
	Usage:³

	Trigger type: Contains
	Recommended trigger: "viva"
	Case sensitive trigger: "false"

	Credits:
*/}}

In that case, I'll need help with Regex. I'll probably ask in YAG's support server

In exported code, if parseArgs is used at least one time, copy the Custom usage text when invalid args provided argument to the Usage: field?

An easier alternative is replace - with \n\t, so basic output now has some indentation. Something like:

//Before:
{{/* CC INFO: #20 - Contains: `viva` - Case sensitive trigger: `false` - Group: `Respuestas` */}}

//After:
{{/* CC INFO:
	ID #20
	Contains: `viva`
	Case sensitive trigger: `false`
	Group: `Respuestas`
*/}}

[x] Code changes have been tested on an instance of YAGPDB, or there are no code changes
[x] I have read and followed the contribution guide

Thanks for the PR!

I thought of adding your minifier here, but maybe It's not necessary: If you saved your CC (in YAG's CP) is because you're already below the 10K limit. What do you think?


How to set the concurrency when using a custom api to access the model?

My api access concurrency is 5, I want to know how dspy handles concurrency when sending requests? Where can I set this parameter?

Set num_threads in classes that accept that parameter like when compiling or evaluating

Set num_threads in classes that accept that parameter like when compiling or evaluating

thanks,but are there any parameters for parallelism if it's just a generation job using pipeline