Reporting Inaccurate Affected Components in GitHub Advisory Database

Dear maintainers of github advisory-database:
We thank GitHub Advisory for providing valuable data to the public. We are utilizing this database for security purposes, and we greatly appreciate the contribution.
During our analysis of the CVE entries, we have identified an issue with the accuracy of the "package" field where the listed affected components does not align with the actual affected components. It can either be incomplete or incorrect.
For instance, in the first entry, CVE-2019-15477 https://github.com/github/advisory-database/blob/6ea0ea8bbfd74f67b7b60b86aac2d0cfe8a6152d/advisories/github-reviewed/2019/08/GHSA-f5f4-m7qp-w6gc/GHSA-f5f4-m7qp-w6gc.json, the database states that the affected component is io.jooby:jooby. However, upon further investigation, we have found that the correct affected component should be org.jooby:jooby, as evidenced by the patch available at https://github.com/jooby-project/jooby/pull/1368/commits/34856a738829d8fedca4ed27bd6ff413af87186f.
Another example is CVE-2018-20094 https://github.com/github/advisory-database/blob/6ea0ea8bbfd74f67b7b60b86aac2d0cfe8a6152d/advisories/github-reviewed/2018/12/GHSA-8j39-fgfp-vxh8/GHSA-8j39-fgfp-vxh8.json, where the same discrepancy occurs. As mentioned in the poc (https://github.com/xuxueli/xxl-conf/issues/61), the ConfController.java line 150 is affected, and this method is located in https://github.com/xuxueli/xxl-conf/blob/6726dfe7979ea6d8fb983771471cde69789de632/xxl-conf-admin/src/main/java/com/xxl/conf/admin/controller/ConfController.java, the affected scope can be more specific: com.xuxueli:xxl-conf-admin.
After thorough examination, we have discovered a total of 64 CVE that contain incorrect or incomplete affected component information. In order to prevent misleading information to end users or false alarms from tools built upon this database, we chose to report these issues to you.
Attached csv document contains 64 records of the erroneous components in the vulnerability database. The fields in the csv file are explained as follows:

Vulnerability ID: Unique CVE identifier/GHSA identifier for each vulnerability entry.
Erroneous Affected Component: The list affected ecosystem and component names in vulnerability database which is either incomplete or incorrect. The format is: “ecosystem__split__component”.
Suggested Component: The correct affected ecosystem and component names according to our inspection where it is corrected either by adding component names or rectifying the existing component names. The format is: “ecosystem__split__component”.
Evidence: Justification for the change.

For further details, please refer to the attached csv file. We hope our findings are beneficial to the community security.
Attached CSV File: Github.Inaccurate.Affected.Components.csv
If you have any questions or require additional information, please do not hesitate to reach out to us. We kindly request your feedback on the accuracy of our analysis and look forward to receiving your input.

@taladrane @KateCatlin @shelbyc
We thank GitHub Advisory for the valuable data provided. While analyzing CVE entries, we found inaccuracies in the "package" field, with affected components not aligning with the actual ones, possibly being incomplete or incorrect.
The timely and accurate vulnerability information can significantly enhance open-source security. We raised this concern several weeks ago, and now we are considering whether using pull requests instead of issues would be more effective(maybe 64 PRs?)

Hi @catch22out, Thank you for bringing these advisories to our attention. At the moment, we're working with reduced staff but can still review each advisory individually. If you're interested in being credited if we find the proposed changes to be accurate, you can make one community contribution pull request for each GHSA. Otherwise, we can go through the CSV when we can and let you know in this issue what we find with respect to each affected product.

Hi @shelbyc,
We've submitted several pull requests, but unfortunately, they haven't been merged into the main branch and the problem of inaccurate components still persists within those GHSA. Consequently, we've shifted our approach to open this issue.
We've also identified inaccuracies in affected components within GitLab. Most of the components we reported were accepted. Recently, we manually pickup the overlap of our reporting between GitHub and GitLab, and we've provided a list of 29 GHSA that have been confirmed by GitLab. The fields "GitLab Feedback" and "GitLab Reference" are introduced to indicate GitLab's acceptance status and acceptance evidence.
We believe this can reduce your workload. If you have any questions or run into any issues, please don't hesitate to contact us. We kindly request your feedback on the accuracy of our analysis and look forward to receiving your input~

hi @catch22out 👋 thank you so much for your patience on this as we work through it. I want to share an update about this work and ask for some additional information on a few advisories. We've updated the following advisories to match your suggestions:

GHSA-f5f4-m7qp-w6gc
GHSA-8j39-fgfp-vxh8
GHSA-6g3c-2mh5-7q6x
GHSA-fmqw-vqh5-cwq9
GHSA-9x5v-8352-244g
GHSA-43fp-vwwg-qgv6
GHSA-mm57-9j6q-rxm2
GHSA-qcgx-crrx-38v5
GHSA-2m8h-fgr8-2q9w
GHSA-h7mf-qrm9-2848
GHSA-q7fr-vqhq-v5xr
GHSA-r6fx-55x3-f9x6
GHSA-h8jc-jmrf-9h8f
GHSA-jh5g-9m4v-9vv9
GHSA-26vr-8j45-3r4w
GHSA-2mp8-qvqm-3xwq
GHSA-92wj-x78c-m4fx
GHSA-pmqq-7wfv-jfff
GHSA-f6f2-pwrj-64h3
GHSA-v6wr-fch2-vm5w
GHSA-9jq9-c2cv-pcrj
GHSA-g644-pr5v-vppf
GHSA-jc7p-5r39-9477
GHSA-6v7w-535j-rq5m
GHSA-ccjp-w723-2jf2
GHSA-793h-6f7r-6qvm + GHSA-9p5g-vg43-mj5r
GHSA-fh32-35w2-rxcc
GHSA-xjrr-xv9m-4pw5
GHSA-344f-f5vg-2jfj
GHSA-pr38-qpxm-g88x
GHSA-jqx5-h2hw-5q4f
GHSA-gfwj-fwqj-fp3v
GHSA-4fv4-cq5v-x45m
GHSA-p63h-7hw8-5cw4
GHSA-vvw4-rfwf-p6hx
GHSA-xhw6-hjc9-679m
GHSA-729f-wvj3-c4pj
GHSA-w69w-jvc7-wjgv
GHSA-5993-wwpg-m92c
GHSA-qhm4-jxv7-j9pq
GHSA-m6mm-q862-j366
GHSA-74qp-233x-p5j8
GHSA-f268-65qc-98vg
GHSA-cqqj-4p63-rrmm
GHSA-3357-829x-m9pr
GHSA-w4fj-ccr6-7pcp
GHSA-c4r5-xvgw-2942
GHSA-p699-3wgc-7h72
GHSA-q2xp-75m7-gv52
GHSA-rpvr-mw7r-25xx
GHSA-grg4-wf29-r9vv
GHSA-wc4x-4gm2-74j8
GHSA-9xrj-439h-62hg
GHSA-42wx-65g4-5cxv
GHSA-484q-784p-8m5h
GHSA-7q8g-gpfp-v8gx
GHSA-pr5m-4w22-8483

The following advisories are ones that we either have additional questions about or currently disagree with your feedback:

GHSA-ffm7-7r8g-77xm: Partially agree - we agree with addingorg.apache.cxf:cxf-rt-management as Snyk included the commit reference: https://security.snyk.io/vuln/SNYK-JAVA-ORGAPACHECXF-564238
GHSA-pc4w-8v5j-29w9: We're unable to find anything related to neo4j-shell on GitHub, including older versions on the neo4j repo. Do you have any additional information about this?
GHSA-mc8v-mgrf-8f4m: while we agree in theory, we can only support modules in Go since users have to update the module to update an associated package.
GHSA-3pp3-77j6-8ph6: Almost agree - changes are made to a file in this pom.xml tree, which corresponds with org.apache.nifi:nifi-web (https://central.sonatype.com/artifact/org.apache.nifi/nifi-web), not org.apache.nifi:nifi-web-security
GHSA-9gcm-f4x3-8jpw: Partially agree - the correct package is only org.springframework:spring-web based on the commit (https://github.com/spring-projects/spring-framework/commit/f64fa3dea10af125d612d3a997aece93d21bc875)
GHSA-3448-vfvv-xp9g: Disagree. The tika-parser-sqlite3-module was introduced in 2021. The vulnerability is more likely affecting tika-parsers as the file SQLite3Parser in versions before 1.20 can be found here.
GHSA-grc3-8q8m-4j7c: Disagree - the only file changed in the fix commit corresponds to org.apache.accumulo:accumulo-master

Hi @catch22out, thank you again for your assistance in finding these errors!
Did this resolve this for you or are there any errors persisting? If it's all resolve, I'll go ahead and close this issue!

Thank you for all your efforts!
You have collectively complete disagree with https://github.com/advisories/GHSA-3448-vfvv-xp9g  and https://github.com/advisories/GHSA-grc3-8q8m-4j7c. For the first one, we agree with your decision, SQLite3Parser also appear in tika-parser-sqlite3-module but not vulnerable.
For the second one, there are still some questions.  The repository is changed and the artifact of accumulo-manager is deleted currently. When roll back to the patched version, the modified vulnerable file is server/manager/src/main/java/org/apache/accumulo/master/MasterClientServiceHandler.java and the pom.xml file in https://github.com/apache/accumulo/blob/64c176eaa29703031e6a4fce9e8d883c6b41424e/server/manager/pom.xml pointed to accumulo-manager.
In all, we would like to express our sincere gratitude to GitHub for your recognition and support of our work. We are also very appreciative of your willingness to take the time to address these issues and look forward to making further contributions to the open-source ecosystem in the future🙂.

Currently, we have discovered some additional inaccuracies in affected components for Ten GHSAs. We also kindly request your attention to these inaccuracies.

hi @catch22out 👋 thank you for the feedback! We've reviewed and agree with your suggestions for those 10 GHSAs and have made the appropriate updates. Following up on CVE-2020-17533 / GHSA-grc3-8q8m-4j7c, where we still disagree, here is our reasoning:

The PR which corresponds to the main branch and the 2.1.0 branch, has MasterClientServiceHandler.java in server/manager. If a user builds the code from the main branch or uses 2.1.0, that's where the file will be.

However, when the fix was backported to 2.0.1 and 1.10.1, the fix was applied to the server/master folder because that is where MasterClientServiceHandler.java is in those versions. The fix commit for 1.10.1 has MasterClientServiceHandler.java in service/master too.
If you look at the vulnerable versions 2.0.0 and 1.10.0, there is no manager subfolder under the server folder, only
master. manager doesn't appear until 2.1.0 . So while the fix appears the manager folder, manager is never actually vulnerable.

https://github.com/apache/accumulo/tree/rel/2.0.0/server
https://github.com/apache/accumulo/tree/rel/1.10.0/server
https://github.com/apache/accumulo/tree/rel/2.1.0/server

Here's the PR that renames master to manager in 2.1.0:

Additionally, MvnRepository says that the name of org.apache.accumulo:accumulo-master is changed to org.apache.accumulo:accumulo-manager in 2.1.0. So manager has no vulnerable versions associated with it.

@catch22out any other questions here or shall we close this issue out?

Thank you for the discussion. I agree with the conclusion of  @taladrane. I believe all aspects of the issue have been addressed. I don't have any further questions at the moment. Let's go ahead and close this issue. Thanks again for your review!


Compute genesis root

Describe the feature
Genesis state root is not computed, but is a mandatory field inside Genesis struct.
This should be computed rather then be required, using genesis_state_root fn, to be in line with Geth behaviour.
It also includes tests for init_genesis.
Additional context
Fixes #896

Atm, i wrote 3 basic tests for init_gensis. While first two passes, the third one success_init_genesis_sepolia doesn't.
computed_genesis: '0x2f7e7be7b9b001d477a981dcd9a4ffe20ff8d90f92e9b0f366cdc32efc789aab',
GENESIS_SEPOLIA: '0x25a5cc106eea7138acab33231d7160d69cb777ee0c2c553fcddf5138993e6dd9'

Ah, this is due to a current bug in how we set up forks for sepolia - will fix

@Rjected
Ok I updated tests removing those genesis_hash setting and rebased on top of latest commit. The problem about sepolia genesis init test seems solved.

looks like there are some conflicts that need to be fixed, otherwise this looks great! tysm!

@Rjected Ty! Rebased, and fixed conflicts 👍🏻

nice! gg


Terminal Text duplicates when backspacing in split-screen mode.

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.59.0
OS Version: Windows 10 Version 20H2

Steps to Reproduce:

Open VSCode in split-screen mode.
Open a new terminal
Try typing out a command
Try backspacing

Here is a screenshot:

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

Weird backspace handling in terminal/pwsh (#127098)

That is not what is happening but hey looks like that is an experimental feature.

I know that a lot of issues get raised on the VSCode repo, but it's been 7 days and this issue is getting worse. I now normally have my windows in this layout: A VSCode Window taking up half my screen space, a Microsoft Edge window takes up a quarter, and another VSCode window takes up another quarter. With the half VSCode window, if I type a command and try backspacing, randomly it just cuts into the path. I don't have a screenshot, but it looks something like this after backspacing: C:git commit rojects/Nexa> git comm-m "commit message".

Hey, yes lots of issues coming in. Sorry for the delay. Could you pls provide a gif of the issue so I can investigate it?

@meganrogge Once it happens again, I'll send a GIF, because it happens randomly.

ok sounds good thanks

bruh I said it happens randomly...

@meganrogge So it has happened again.

Try changing the front slash (/) to back slash ()

@TarunavBA how is that gonna work?

@vscode-triage-bot how about you re-open the issue, please?


build errors on gcc-11.1.0 and clang++ 13.0.0  (fix included)

I am hitting plenty of errors about unknown numeric_limits
/data/sources/switch/nstool/deps/libtoolchain/include/tc/crypto/detail/XtsModeImpl.h: In member function ‘void tc::crypto::detail::XtsModeImpl<BlockCipher>::incr_tweak_be(byte_t*, uint64_t)’:
/data/sources/switch/nstool/deps/libtoolchain/include/tc/crypto/detailXtsModeImpl.h:275:51: error: ‘numeric_limits’ is not a member of ‘std’
  275 |                         uint64_t remaining = std::numeric_limits<uint64_t>::max() - word;
      |

the fix is to add
include <limits>

to deps/libtoolchain/include/tc/types.h
that seems to fix the build.

Thanks for pointing this out. I'll get onto this soon.


Provide parameterized test

as recommended by Sonar
https://sonarcloud.io/project/issues?id=camel-lsp-server&issues=AXYZ0sg4VEJUvx7Dhpf-&open=AXYZ0sg4VEJUvx7Dhpf-

Interesting approach! May have to learn more about the Jupiter framework.


Write short units of code

Separate functions, which have more that 60 lines of code, to smaller functions to pass Better Code Hub check.
Functions with more than 60 lines of code:

stopwords.py
shapes.py
trainer.py
hyperscan.py

Changed shapes.py


 Please help me .
Thank you .

Hi @mama110,
I was facing this error too, but while training on my own dataset. The problem for me was an annotation file without any object in it.
Don't know if this will help you but, here is the script I used to check my annotations files :
import argparse
import sys
import cv2
import os

import os.path          as osp
import numpy            as np

if sys.version_info[0] == 2:
    import xml.etree.cElementTree as ET
else:
    import xml.etree.ElementTree  as ET

parser    = argparse.ArgumentParser(
            description='Single Shot MultiBox Detector Training With Pytorch')
train_set = parser.add_mutually_exclusive_group()

parser.add_argument('--root', help='Dataset root directory path')

args = parser.parse_args()

CLASSES = (  # always index 0
    'aeroplane', 'bicycle', 'bird', 'boat',
    'bottle', 'bus', 'car', 'cat', 'chair',
    'cow', 'diningtable', 'dog', 'horse',
    'motorbike', 'person', 'pottedplant',
    'sheep', 'sofa', 'train', 'tvmonitor')

annopath = osp.join('%s', 'Annotations', '%s.{}'.format("xml"))
imgpath  = osp.join('%s', 'JPEGImages',  '%s.{}'.format("jpg"))

def vocChecker(image_id, width, height, keep_difficult = False):
    target   = ET.parse(annopath % image_id).getroot()
    res      = []

    for obj in target.iter('object'):

        difficult = int(obj.find('difficult').text) == 1

        if not keep_difficult and difficult:
            continue

        name = obj.find('name').text.lower().strip()
        bbox = obj.find('bndbox')

        pts    = ['xmin', 'ymin', 'xmax', 'ymax']
        bndbox = []

        for i, pt in enumerate(pts):

            cur_pt = int(bbox.find(pt).text) - 1
            # scale height or width
            cur_pt = float(cur_pt) / width if i % 2 == 0 else float(cur_pt) / height

            bndbox.append(cur_pt)

        print(name)
        label_idx =  dict(zip(CLASSES, range(len(CLASSES))))[name]
        bndbox.append(label_idx)
        res += [bndbox]  # [xmin, ymin, xmax, ymax, label_ind]
        # img_id = target.find('filename').text[:-4]
    print(res)
    try :
        print(np.array(res)[:,4])
        print(np.array(res)[:,:4])
    except IndexError:
        print("\nINDEX ERROR HERE !\n")
        exit(0)
    return res  # [[xmin, ymin, xmax, ymax, label_ind], ... ]

if __name__ == '__main__' :

    i = 0

    for name in sorted(os.listdir(osp.join(args.root,'Annotations'))):
    # as we have only one annotations file per image
        i += 1

        img    = cv2.imread(imgpath  % (args.root,name.split('.')[0]))
        height, width, channels = img.shape

        print("path : {}".format(annopath % (args.root,name.split('.')[0])))
    print("Total of annotations : {}".format(i))

@Eralaf It worked like a charm. Thanks a lot!
FYI, you forgot to call vocChecker in main()
I pasted the following code after height, width, channels = img.shape:
res = vocChecker((args.root, name.split('.')[0]), height, width)

@Eralaf It help me a lot

facing same error,
trying to train my own dataset and annotations are json files...
how to solve the error

@Eralaf  i have tried your method on pycharm,but error shows up :
Traceback (most recent call last):
File "C:/Users/Administrator/Desktop/ssd.pytorch-master/error.py", line 76, in
for name in sorted(os.listdir(osp.join(args.root,'Annotations'))):
TypeError: expected str, bytes or os.PathLike object, not NoneType.
so could tell me how to solve the problem ,thanks!
(my torch version is 1.0)

@ankitksharma @guiw629
i have tried the method on pycharm,but error shows up :
Traceback (most recent call last):
File "C:/Users/Administrator/Desktop/ssd.pytorch-master/error.py", line 76, in
for name in sorted(os.listdir(osp.join(args.root,'Annotations'))):
TypeError: expected str, bytes or os.PathLike object, not NoneType.
so could tell me how to solve the problem ,thanks!
(my torch version is 1.0)

@barcahoo Did you call the script with the root argument ?
Let's say the script I used to check my annotations files is called check.py :
python check.py --root="Dataset root directory path"

In this root directory there must be an "Annotations" folder and  a "JPEGImages" folder :)
@charan1561 it's a bit late, sorry.
When you say your annotations are same as coco annotations, do you mean your dataset is a subset from the cocodataset or your annotations files have the same structure ?
Did you try checking your annotations in case there are empty ones (without objects) ? If so, the idea is to not use them :)

@Eralaf  thanks a lot ! it works

@Eralaf  thanks a lot ! it works

@Eralaf thanks! it works, but what puzzies me is that, I added a print log to the top of this sentence, like
print("target {}".format(target)) img, boxes, labels = self.transform(img, target[:, :4], target[:, 4])
why didn't it print out the target of empty, instead, it printed a target with shape (1, 5)

I have this error too，and i use @Eralaf method to check xml
finally，i Found the cause of the mistake
if not self.keep_difficult and difficult:                 continue
Original code dose not keep difficulte instances in training make xml parse have no object of  labels and boxes，so run to
img, boxes, labels = self.transform(img, target[:, :4], target[:, 4])
have IndexError: too many indices for array

I think that will  happend when there are not objects in the xml file,because this code does not process this condition

I think that will happend when there are not objects in the xml file,because this code does not process this condition

I feel the same way, but how to solve it ?

I think that will happend when there are not objects in the xml file,because this code does not process this condition

I think that will happend when there are not objects in the xml file,because this code does not process this condition

I feel the same way, but how to solve it ?

I think that will happend when there are not objects in the xml file,because this code does not process this condition

Hi @mama110,
I was facing this error too, but while training on my own dataset. The problem for me was an annotation file without any object in it.
Don't know if this will help you but, here is the script I used to check my annotations files :
import argparse
import sys
import cv2
import os

import os.path          as osp
import numpy            as np

if sys.version_info[0] == 2:
    import xml.etree.cElementTree as ET
else:
    import xml.etree.ElementTree  as ET

parser    = argparse.ArgumentParser(
            description='Single Shot MultiBox Detector Training With Pytorch')
train_set = parser.add_mutually_exclusive_group()

parser.add_argument('--root', help='Dataset root directory path')

args = parser.parse_args()

CLASSES = (  # always index 0
    'aeroplane', 'bicycle', 'bird', 'boat',
    'bottle', 'bus', 'car', 'cat', 'chair',
    'cow', 'diningtable', 'dog', 'horse',
    'motorbike', 'person', 'pottedplant',
    'sheep', 'sofa', 'train', 'tvmonitor')

annopath = osp.join('%s', 'Annotations', '%s.{}'.format("xml"))
imgpath  = osp.join('%s', 'JPEGImages',  '%s.{}'.format("jpg"))

def vocChecker(image_id, width, height, keep_difficult = False):
    target   = ET.parse(annopath % image_id).getroot()
    res      = []

    for obj in target.iter('object'):

        difficult = int(obj.find('difficult').text) == 1

        if not keep_difficult and difficult:
            continue

        name = obj.find('name').text.lower().strip()
        bbox = obj.find('bndbox')

        pts    = ['xmin', 'ymin', 'xmax', 'ymax']
        bndbox = []

        for i, pt in enumerate(pts):

            cur_pt = int(bbox.find(pt).text) - 1
            # scale height or width
            cur_pt = float(cur_pt) / width if i % 2 == 0 else float(cur_pt) / height

            bndbox.append(cur_pt)

        print(name)
        label_idx =  dict(zip(CLASSES, range(len(CLASSES))))[name]
        bndbox.append(label_idx)
        res += [bndbox]  # [xmin, ymin, xmax, ymax, label_ind]
        # img_id = target.find('filename').text[:-4]
    print(res)
    try :
        print(np.array(res)[:,4])
        print(np.array(res)[:,:4])
    except IndexError:
        print("\nINDEX ERROR HERE !\n")
        exit(0)
    return res  # [[xmin, ymin, xmax, ymax, label_ind], ... ]

if __name__ == '__main__' :

    i = 0

    for name in sorted(os.listdir(osp.join(args.root,'Annotations'))):
    # as we have only one annotations file per image
        i += 1

        img    = cv2.imread(imgpath  % (args.root,name.split('.')[0]))
        height, width, channels = img.shape
        print("path : {}".format(annopath % (args.root,name.split('.')[0])))
        res = vocChecker((args.root, name.split('.')[0]), height, width)
    print("Total of annotations : {}".format(i))

Excuse me , When we check out these files that are empty, what do we do with them? delete?Hope to receive our reply !

From the beginning to the end of the loss has always been equal to 2.000, what is the reason


Configurable / improved error handling

Currently, a number of errors in the backend get handled via calls to abort(); this is fine if we're executing an interpreter binary, as there's nothing more we can do in this situation other than crash the interpreter. However, if we're executing as a library via the Python or Haskell bindings, this causes a problem - abort() will crash the parent process as well as the LLVM backend, which precludes any kind of recovery.
We would like the parent processes that call into the LLVM backend to be able to recover from errors; it's not implausible that they permit some form of user input that can produce backend errors that aren't feasible to catch in the client.
This originally cropped up in #622, which has its own fix, but we should address the error handling problem as well.
I suspect the solution here is to expose an API to install custom error handlers; the default versions of these can just call abort() as previous. Something like:
void set_llvm_backend_error_handler(void (*handler)(char const*));

@Baltoli investigate how tricky it will be to return control flow to the calling binding; what are the ergonomics like?

@sskeirik identifies that this is also useful in the "embedded custom main" scenario.


Binaries differ with same code but multiple build runs (CMake, esp as library) (IDFGH-3257)

Environment

Development Kit: none
Module or chip used: ESP32-WROOM-32
IDF version:

v3.3.2-107-g722043f73
v4.0-327-gfe67bedee (edit: also reproduced it with this version)

Build System: CMake
Compiler version:

xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) 5.2.0 (on esp idf v3.3.2)
xtensa-esp32-elf-gcc (crosstool-NG esp-2019r2) 8.2.0 (on esp idf v4.0)

Operating System: Linux
Using an IDE?: No
Power Supply: USB

Problem Description
Compiled binaries differ between different builds even if nothing changed in the code. This happens at least when esp-idf is used as a library (I have not tried other configurations).
First of all this is highly worrying. Binaries should be reproducible and should not randomly vary. Secondly, this is quite critical to us as we use an external PKI to sign the binaries for secure boot.
Expected Behavior
The hashes between the binaries (generated with two independent fresh builds) without any code changes should be same.
Actual Behavior
The hashes differ
Steps to reproduce
I can reproduce this with a freshly cloned repository with the example build_system/cmake/idf_as_lib with the following commands:
cd examples/build_system/cmake/idf_as_lib
./build.sh
mv build build.bkp
rm sdkconfig
./build.sh
sha256sum build/idf_as_lib.bin build.bkp/idf_as_lib.bin
759e17e59ac3063a3e2ae167f48cf1f1e825bc29d9757490b0ae2df13c2e3f78  build/idf_as_lib.bin
f1bf34285a6975f2c2514cfb287dbdbf616624a70ea4c7b11eeddf05e7851117  build.bkp/idf_as_lib.bin
A recursive diff between the two build folders show that even the sublibraries already differ:
diff -qr build build.bkp/
Files build/bootloader/bootloader.bin and build.bkp/bootloader/bootloader.bin differ
Files build/bootloader/bootloader.elf and build.bkp/bootloader/bootloader.elf differ
Files build/bootloader/CMakeFiles/CMakeOutput.log and build.bkp/bootloader/CMakeFiles/CMakeOutput.log differ
Files build/bootloader/esp-idf/bootloader_support/CMakeFiles/idf_component_bootloader_support.dir/src/bootloader_init.c.obj and build.bkp/bootloader/esp-idf/bootloader_support/CMakeFiles/idf_component_bootloader_support.dir/src/bootloader_init.c.obj differ
Files build/bootloader/esp-idf/bootloader_support/libbootloader_support.a and build.bkp/bootloader/esp-idf/bootloader_support/libbootloader_support.a differ
Files build/bootloader/esp-idf/efuse/libefuse.a and build.bkp/bootloader/esp-idf/efuse/libefuse.a differ
Files build/bootloader/esp-idf/log/liblog.a and build.bkp/bootloader/esp-idf/log/liblog.a differ
Files build/bootloader/esp-idf/main/libmain.a and build.bkp/bootloader/esp-idf/main/libmain.a differ
Files build/bootloader/esp-idf/micro-ecc/libmicro-ecc.a and build.bkp/bootloader/esp-idf/micro-ecc/libmicro-ecc.a differ
Files build/bootloader/esp-idf/soc/libsoc.a and build.bkp/bootloader/esp-idf/soc/libsoc.a differ
Files build/bootloader/esp-idf/spi_flash/libspi_flash.a and build.bkp/bootloader/esp-idf/spi_flash/libspi_flash.a differ
Files build/CMakeFiles/CMakeOutput.log and build.bkp/CMakeFiles/CMakeOutput.log differ
Files build/esp-idf/app_trace/libapp_trace.a and build.bkp/esp-idf/app_trace/libapp_trace.a differ
Files build/esp-idf/app_update/CMakeFiles/idf_component_app_update.dir/esp_app_desc.c.obj and build.bkp/esp-idf/app_update/CMakeFiles/idf_component_app_update.dir/esp_app_desc.c.obj differ
Files build/esp-idf/app_update/libapp_update.a and build.bkp/esp-idf/app_update/libapp_update.a differ
Files build/esp-idf/bootloader_support/libbootloader_support.a and build.bkp/esp-idf/bootloader_support/libbootloader_support.a differ
Files build/esp-idf/cxx/libcxx.a and build.bkp/esp-idf/cxx/libcxx.a differ
Files build/esp-idf/driver/libdriver.a and build.bkp/esp-idf/driver/libdriver.a differ
Files build/esp-idf/efuse/libefuse.a and build.bkp/esp-idf/efuse/libefuse.a differ
Files build/esp-idf/esp32/libesp32.a and build.bkp/esp-idf/esp32/libesp32.a differ
Files build/esp-idf/espcoredump/libespcoredump.a and build.bkp/esp-idf/espcoredump/libespcoredump.a differ
Files build/esp-idf/esp_event/libesp_event.a and build.bkp/esp-idf/esp_event/libesp_event.a differ
Files build/esp-idf/esp_ringbuf/libesp_ringbuf.a and build.bkp/esp-idf/esp_ringbuf/libesp_ringbuf.a differ
Files build/esp-idf/ethernet/libethernet.a and build.bkp/esp-idf/ethernet/libethernet.a differ
Files build/esp-idf/freertos/libfreertos.a and build.bkp/esp-idf/freertos/libfreertos.a differ
Files build/esp-idf/heap/libheap.a and build.bkp/esp-idf/heap/libheap.a differ
Files build/esp-idf/log/liblog.a and build.bkp/esp-idf/log/liblog.a differ
Files build/esp-idf/lwip/liblwip.a and build.bkp/esp-idf/lwip/liblwip.a differ
Files build/esp-idf/mbedtls/libmbedtls.a and build.bkp/esp-idf/mbedtls/libmbedtls.a differ
Files build/esp-idf/mbedtls/mbedtls/library/libmbedcrypto.a and build.bkp/esp-idf/mbedtls/mbedtls/library/libmbedcrypto.a differ
Files build/esp-idf/mbedtls/mbedtls/library/libmbedtls.a and build.bkp/esp-idf/mbedtls/mbedtls/library/libmbedtls.a differ
Files build/esp-idf/mbedtls/mbedtls/library/libmbedx509.a and build.bkp/esp-idf/mbedtls/mbedtls/library/libmbedx509.a differ
Files build/esp-idf/micro-ecc/libmicro-ecc.a and build.bkp/esp-idf/micro-ecc/libmicro-ecc.a differ
Files build/esp-idf/newlib/libnewlib.a and build.bkp/esp-idf/newlib/libnewlib.a differ
Files build/esp-idf/nvs_flash/libnvs_flash.a and build.bkp/esp-idf/nvs_flash/libnvs_flash.a differ
Files build/esp-idf/pthread/libpthread.a and build.bkp/esp-idf/pthread/libpthread.a differ
Files build/esp-idf/smartconfig_ack/libsmartconfig_ack.a and build.bkp/esp-idf/smartconfig_ack/libsmartconfig_ack.a differ
Files build/esp-idf/soc/libsoc.a and build.bkp/esp-idf/soc/libsoc.a differ
Files build/esp-idf/spi_flash/libspi_flash.a and build.bkp/esp-idf/spi_flash/libspi_flash.a differ
Files build/esp-idf/tcpip_adapter/libtcpip_adapter.a and build.bkp/esp-idf/tcpip_adapter/libtcpip_adapter.a differ
Files build/esp-idf/vfs/libvfs.a and build.bkp/esp-idf/vfs/libvfs.a differ
Files build/esp-idf/wpa_supplicant/libwpa_supplicant.a and build.bkp/esp-idf/wpa_supplicant/libwpa_supplicant.a differ
Files build/esp-idf/xtensa-debug-module/libxtensa-debug-module.a and build.bkp/esp-idf/xtensa-debug-module/libxtensa-debug-module.a differ
Files build/idf_as_lib.bin and build.bkp/idf_as_lib.bin differ
Files build/idf_as_lib.elf and build.bkp/idf_as_lib.elf differ
Code to reproduce this issue
Standard example in examples/build_system/cmake/idf_as_lib without any changes.

Hi @shangl,
Thanks for the detailed bug report and letting us know about this. Reproducible builds has been a goal for ESP-IDF for a while and we're actually closer to this than ever in the CMake-based build system, but it's still not immediately achievable (as made clear in your report). However it should be achievable very soon.
There are a few factors involved, I'll try to break them down individually and explain what can be done about them:

Files build/esp-idf/app_update/CMakeFiles/idf_component_app_update.dir/esp_app_desc.c.obj and build.bkp/esp-idf/app_update/CMakeFiles/idf_component_app_update.dir/esp_app_desc.c.obj differ
Files build/idf_as_lib.bin and build.bkp/idf_as_lib.bin differ
Files build/idf_as_lib.elf and build.bkp/idf_as_lib.elf differ

By default, ESP-IDF apps have the build time & date compiled into an "app description" structure which can be accessed via an API.
This can be disabled by turning off this config flag:
https://docs.espressif.com/projects/esp-idf/en/v3.3.2/api-reference/kconfig.html#config-app-compile-time-date

Files build/bootloader/bootloader.bin and build.bkp/bootloader/bootloader.bin differ
Files build/bootloader/bootloader.elf and build.bkp/bootloader/bootloader.elf differ
Files build/bootloader/esp-idf/bootloader_support/CMakeFiles/idf_component_bootloader_support.dir/src/bootloader_init.c.obj and build.bkp/bootloader/esp-idf/bootloader_support/CMakeFiles

The bootloader logs the time that it was compiled as part of the default startup sequence.
One workaround for this is to change the bootloader log level to Warning, Error or None - meaning no Info level logs are compiled in.
Another workaround is to patch the log line out in the IDF source:
diff --git a/components/bootloader_support/src/bootloader_init.c b/components/bootloader_support/src/bootloader_init.c
index 17d0fb161c..2a44a691ac 100644
--- a/components/bootloader_support/src/bootloader_init.c
+++ b/components/bootloader_support/src/bootloader_init.c
@@ -148,7 +148,7 @@ static esp_err_t bootloader_main()
     wdt_reset_check();
     ESP_LOGI(TAG, "ESP-IDF %s 2nd stage bootloader", IDF_VER);

-    ESP_LOGI(TAG, "compile time " __TIME__ );
+    //ESP_LOGI(TAG, "compile time " __TIME__ );
     ets_set_appcpu_boot_addr(0);

 #ifdef CONFIG_BOOTLOADER_WDT_ENABLE

We'll add a config option so it's possible to disable this log line properly.

even the sublibraries already differ

Archive files (.a) are a simple archive format which includes a modification timestamp in the header. So in most of these cases all of the .obj files in the archive have the same contents, and are included in the same order, but the timestamps are different.
ar provides a "deterministic mode" flag which sets all timestamps and other metadata to zero. CMake doesn't have native support for this, but it looks like we can patch it in without too much fuss.
For now, if all of the .obj files are the same then the actual library data should be the same as well and this shouldn't cause any problems for the final binary contents.

There is one other barrier to reproducible builds which is not shown in your diffs, I encountered it in about 50% of my tests just now. We recently fixed a bug on master branch (ec2e666a6ef7d9841393f5f9996472cd1df11ab1) where the linker script generated by the ESP-IDF build system contained some non-determinism in clause ordering. Ordering differences in the linker script can lead to objects being linked in a different order, which leads to different binaries. If you see two different .ld files in two subsequent builds, this is the likely root cause.
This fix hasn't been backported to the release/v4.0 or release/v3.3 branches yet, but we'll look at doing so. Once it's fixed there, with the two modifications shown above then clean builds of the same project using all the same paths, the same source files and configuration file should produce the same .elf and .bin files.

PS Just in case you also encounter this while doing further experiments, I ran into an unexpected behaviour in the idf_as_lib example starting from IDF v4.0 - the top-level CMakeLists.txt specifies that the sdkconfig file is located in the build directory, but the build-esp32.sh script deletes and recreates the build directory each time so the project always builds with default config. This is obviously sub-optimal, will fix the example so the sdkconfig file is located in the main project directory similar to the v3.x behaviour.

@projectgus : thank you very much for the quick response. Do you know approximately when the fixes will be available?

Hi @shangl,
Sorry for the slow reply the second time around!
Regarding the linker script non-determinism, this has been backported to release/v3.3 branch in commit ccbca4570926b11083c14e3c1f24937e530a23f2 so you can use this now. The fix will be in the v3.3.3 stable bugfix release when this is available.
The release/v4.0 branch is currently frozen as v4.0.1 is about to be released, but it will be merged ASAP after the v4.0.1 release and included in the next v4.0.2 bugfix release.
Regarding the other tweaks - there is no single config option yet to get reproducible builds but if you set all the config items steps shown above (or optionally comment the one log line in the bootloader) then you can achieve the same goal. Once a single config option is added to control this then this GitHub issue will be updated.

Hi @projectgus , thank you very much, the fixes look good.

Hi @shangl ,
Glad you're able to make it work. If it's OK with you then I'd prefer to keep this open until we have a documented way to achieve this.
Angus

@projectgus : sure, no problem.
I discovered another issue and I am not sure if whether it connects to this problem, should be different ticket or is a bug at all: I can also see that when changing the flash size, also the application binary starts varying (even with all the adjustments mentioned above). Is this desired / expected? If yes, what is the reason for a different binary when just the flash size is changed in sdkconfig?

@projectgus : it looks like this issue was fixed e.g. for version 4.2.1 but it came back in version 4.3. could this be fixed there too?

@shangl does this issue occur in 4.3 with the same steps to reproduce as described in your original post?

@shangl Thanks for reporting, would you please help share if any updates? Thanks.

I think the issue in release/v4.3 branch will be fixed by https://github.com/espressif/esp-idf/commit/404ee09181375b39d91a8b198bba0afc43d1b21b. @shangl if you are still working on this, please try to update to that commit, let us know if the behavior is still not correct.

Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates.


$PSScriptRoot and $PSCommandPath are not populated with Start-Job -FilePath <filepath>

Steps to reproduce
script.ps1:
echo $PSCommandPath
echo $PSScriptRoot

Run this in terminal:
Start-Job -FilePath .\script.ps1 | Receive-Job -Wait

Expected behavior
<path-to-script.ps1>
<path-to-scrpt.ps1-dir>

Actual behavior

(2 empty lines)
Environment data

PSVersion                      7.2.0-preview.3
PSEdition                      Core
GitCommitId                    7.2.0-preview.3
OS                             Microsoft Windows 10.0.19042
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

The explanation for the current behavior (which is definitely debatable) is implied by this line from the docs:

When you use this parameter, PowerShell converts the contents of the specified script file to a script block and runs the script block as a background job.

That is, the background job doesn't execute the script file given, but its contents, as a script block - and such a script block has no information about where it ultimately came from.
While this behavior makes perfect sense in the context of remoting (with Invoke-Command -ComputerName $someComputer -FilePath ...), it arguably makes better sense to let the background job - which by definition runs on  the same machine and can therefore access the specified file  - execute the script file directly, so as not to lose invocation context.

Don't think this should be closed, as the issue is not really resolved. @iSazonov

The issue is not blocked and anyone can continue to discuss.
This is a behavior "by-design" and it needs huge reason to change it. This kind of anonymity can be beneficial or even security-related.

I don't think security concerns are in the picture here, and my guess is that this was done more for the convenience of sharing an implementation with / behavioral symmetry with Invoke-Command -FilePath
That said, @MatejKafka, the problem is easily worked around by using a script block instead:
Start-Job { .\script.ps1 } | Receive-Job -Wait

Note that in Windows PowerShell this won't necessarily work, because background jobs there don't inherit the caller's location, but in PowerShell Core that isn't a problem anymore.


Putting back the swagger-clinet

Not sure exactly what happened, there missing files.
putting back.

@amastbau we need to sign the commit before merging this PR .
@ibragins can help in signing if needed.


Support for vaadin v 24.5?

Hello!
I tried naively bumping the vaadin version up on this project to 24.5.7 and it blew up.  Would it be a major refactor to support it?
Many thanks for all your work!

Are you getting this error?:
java.lang.NoClassDefFoundError: org/apache/commons/codec/digest/DigestUtils
	at com.vaadin.copilot.analytics.AnalyticsClient.<init>(AnalyticsClient.java:44) ~[copilot-24.6.0.jar:na]
	at com.vaadin.copilot.analytics.AnalyticsClient.getInstance(AnalyticsClient.java:59) ~[copilot-24.6.0.jar:na]
	at com.vaadin.copilot.CopilotIndexHtmlLoader.serviceInit(CopilotIndexHtmlLoader.java:45) ~[copilot-24.6.0.jar:na]
	at com.vaadin.flow.server.VaadinService.lambda$init$0(VaadinService.java:271) ~[flow-server-24.6.0.jar:24.6.0]

Some version conflict introduced by copilot 🤔 Probably excluding it is the easiest solution.

It was quite a fight, but this commit seems to get at least to some extent working project:
https://github.com/mstahv/flow-with-jwt-authentication/commit/f19ffec17f325266dfa1b8c23e11cf6a9cbe3f50

There is still a part that is using a deprecated version of Spring Security API, but fixing that ought to be easy.

Confirmed!  It works great.  Many thanks for your work on this... I know housekeeping can be dull, but this really helps me out.
In terms of feature requests, does your team like to receive those in GitHub Issues?  If not, I can add some comments on the blog.  They're mostly related to the replace cookies in every request change that occurred in 24.3.9


“ledger_entry” without default parameters not returning "invalidParams" error.

Issue Description
“ledger_entry” with one or more fields are specified incorrectly, or one or more required fields are missing i.e., without index, account_root , directory, offer, ripple_state, check, escrow, payment_channel, deposit_preauth, ticket should result in invalidParams error.
Steps to Reproduce
{
"method": "ledger_entry",
"params": [
{
"ledger_index": "validated"
}
]
}
Expected Result
invalidParams
Actual Result
unknownOption

should malformedRequest, malformedAddress, malformedCurrency also be invalidParams ?

What is wrong with throwing unknowOption error ?
Accord to link,
unknownOption - The fields provided in the request did not match any of the expected request formats.
invalidParams - One or more fields are specified incorrectly, or one or more required fields are missing.


Updator problems

Unless I'm just very impatient, the updater doesn't seem to work. It notifies me of an update just fine, and then downloads it. It asks me if I want to clear the folder, and I say yes. After that it just sits at "Verified Update Package" for a while, with the loading bar still moving, and the 'loading' mouse icon when you mouse over it. Apologies for no screenshots, but the folder's empty, and I can only download the updated version.
UPDATE: Tried it on another computer and it worked flawlessly. Only thing that changed was the OS (worked on win7, not on win8.) Sorry if that's not very helpful.

I think I know which problem you mention.
Did you update from ver. <IP_ADDRESS>?

Yes, from <IP_ADDRESS> to <IP_ADDRESS>

Well, in <IP_ADDRESS> there was a problem which could occure on some systems. If the automatic patch fails you have to download and replace the files from the update package manually. Path: http://j.mp/1NqSo2d
However, this bug should be fixed in ver. <IP_ADDRESS> and later. Sorry for the trouble.

It's okay, glad it got fixed! :) /play tada

Thank you very much. :)


WG: Screens animate out way too quickly to be readable

@mattwbenson you mean the countdowns are too fast?

No, well, yes, they are, but I mean, like, the scoreboards and all that appear and disappear way too quickly, especially if there are only a few clicks.

Like, there's something going wrong with the clickmap, but it disappeared too quickly for me to figure out what it was.


docs: Revert "docs: fixing links"

Reverts dfinity/sdk#2410
The 'fix' here breaks the build in the portal repo. Will revisit how to deal with this.

False alarm, apologies for the inconvenience!


Latest release does not open text file on double click in Windows

VSCode Version: Code 1.6.0 (e52fb0bc87e6f5c8f144e172639891d8d8c9aa55, 2016-10-10T18:37:40.189Z)
OS Version: Windows_NT ia32 6.1.7601
Extensions: DotJoshJohnson.xml-1.6.0, ms-vscode.csharp-1.4.1

This was working fine right before I installed the latest updates.
Steps to Reproduce:

Double click a .txt file with the default program set to use VS Code
VS Code opens but the file does not display

Might this be due to the experiments of opening vscode empty?

I can't reproduce. Is there any information in the Developer Tools console? What actually happens in Code? Can you share a screenshot?

Well I seem to have resolved it, here's how:  I right clicked a .txt file and chose Open With -> Choose Default Program.  Text files were already set to use Code as the default program on my machine, but I selected to use VS Code again.  The file opened in Code and the contents of the .txt file displayed.  I then closed Code and reopened the file, this time by double-clicking (which was not working before).  It now opens and displays the contents of the file.
To answer your question, prior to doing what I described above, I would double click a .txt file and VS Code would open, but it would not open the file.  I saw no errors displayed, and I did not get a chance to look at the developer tools console.  It was as if I just clicked on code.exe by itself.

Strange. I'm glad you fixed it though. 👍

FYI I had the same issue since last update and fixed it the same way

Same problem, same fix.
After Visual Studio Code update, double clicking a txt file would open code but wouldn't load the text file. Re-selecting vs code as the default program fixed the issue and double clicking a txt file now not only opens vs code but also loads the txt file.


Removing dead code from Interactive Beam.

Seems that no one has been referencing PipelineAnalyzer since BEAM-8016.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @KevinGG
Is pipeline_analyzer safe to remove? Seems no one's referencing it since pipeline_instrument came into place.
display_manager.py is also not referenced anymore, but I am not sure about this one since there are more recent changes to that file.

R: @KevinGG
Is pipeline_analyzer safe to remove? Seems no one's referencing it since pipeline_instrument came into place.

Yes, it's safe to remove. Because we were using a different approach to decide what to cache and what to execute to resolve the re-execution/out-of-order execution issues in a REPL environment, we moved away from pipeline_analyzer.

display_manager.py is also not referenced anymore, but I am not sure about this one since there are more recent changes to that file.

We can leave it as it is. Though we started using pipeline_graph directly, the display manager did provide a great abstraction above the underlying os and installed executable dependencies. We could extend it into doing other things in the future.
Thank you for being diligent and clean up the code base!

Thanks for the review!

Rerun test

Run tests

It seems that you need to close then open the PR to trigger Python tests / Python Unit Tests (macos-latest, 3.6, py36) (pull_request).
But I find the test fails all the time for my other PRs. Let's add a committer and see if they can help us.

Is this something that happens often? Do you know how we can resolve this?

I think we sometimes ignore this kind of errors.
Let me add Pablo as the reviewer.
Hi @pabloem, could you please take a look at this PR that cleans up dead code?

Run Python PreCommit

Thanks @KevinGG @qinyeli ! We can merge as is.

Thank you!


[Shared] Markdown doesn't respect Unicode space

see discussion in #3342.

Could we broaden the scope of the issue a bit to include building a tokenizer?
Our MarkDwon parser would benefit from having a separate tokenizer. we could roll up all of the tokenizing task including UTF-8 support in one place.
Having this task as one of the requirements for building tokenizer would be great. Any thought @paulcam206, @RebeccaAnne , and @shalinijoshi19 ?

yeah, that's basically what this is going to require -- we need to tokenize to be able to handle multibyte chars :)

Let's track this via #1984


Issue with /docs/tutorials/kubernetes-basics/cluster-interactive/

I'm trying interactive tutorials on https://kubernetes.io/docs/tutorials/kubernetes-basics/cluster-interactive/
When I push "Start scenario" button, it shows me raccoon in the mask that asks for cookies.
Does it mean that service is down?

Cookies have to be enabled for you to use the interactive tutorial :(


Adding new routes or changing existing is very slow

Describe the bug
Adding new routes or changing existing is very slow. Takes sometime to reflect changes.
To Reproduce
Create a new route or update an existing route.
getambassador.io/config: |
  ---
  apiVersion: ambassador/v1
  kind:  Mapping
  name: test-service-mapping
  prefix: /services/test-service/
  service: test-service.services:8080
  rewrite: /services/test-service/

(Rewrite is correct, my service is running under context)
Expected behavior
Expect the route to take effect immediately.
Versions (please complete the following information):

Ambassador: 0.51.2
Kubernetes environment: AWS(Kubespray)
Version. v1.11.5

Additional context
Also if i change the route from  /services/test-service/ to  /services/testservice/ it takes time to reflect.
I enabled debug logs and i get 404 for few requests. After sometime the url is available.
ACCESS [2019-03-15T20:37:14.589Z] "GET /services/test-service/ HTTP/1.1" 404 NR 0 0 0 - "<IP_ADDRESS>,<IP_ADDRESS>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36" "bda56d69-8961-4916-a020-c448feff9004" "abc.com:8443" "-"
ACCESS [2019-03-15T20:37:15.451Z] "GET /services/test-service/ HTTP/1.1" 404 NR 0 0 0 - "<IP_ADDRESS>,<IP_ADDRESS>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36" "92df5cc6-60fe-42d8-afca-c9b0407edd7a" "internal-kube-aws-preprod2-internal-405356190.us-east-1.elb.amazonaws.com:8443" "-"

Additional info. After ingress is updated, the services is available after 15 secs. Is it a normal behavior?
2019/03/18 17:58:22 SYNC: python3 /ambassador/post_update.py http://localhost:35823/api/snapshot/8
2019-03-18 17:58:22 diagd 0.51.2 [P62TThreadPoolExecutor-0_4] INFO: Update requested from http://localhost:35823/api/snapshot/8
2019-03-18 17:58:22 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: copying configuration from http://localhost:35823/api/snapshot/8 to /ambassador/snapshots/snapshot-tmp.yaml
2019-03-18 17:58:34 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: successfully validated the resulting envoy configuration, continuing...
2019-03-18 17:58:34 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: rotating snapshots for snapshot 8
2019-03-18 17:58:34 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: saving Envoy configuration for snapshot 8
2019-03-18 17:58:34 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: running 'sh /ambassador/kick_ads.sh 46'
KICK: kicking ambex
2019-03-18 17:58:34 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: configuration updated
time="2019-03-18T17:58:34Z" level=info msg="Loaded file /ambassador/envoy/envoy.json"
time="2019-03-18T17:58:34Z" level=info msg="Pushing snapshot v8"
[2019-03-18 17:58:34.975][000114][info][upstream] [source/server/lds_api.cc:80] lds: add/update listener 'ambassador-listener-80'
2019-03-18 17:58:35 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: Scout reports {"latest_version": "0.51.2", "application": "ambassador", "cached": true, "timestamp":<PHONE_NUMBER>.873044}
2019-03-18 17:58:35 diagd 0.51.2 [P62TAmbassadorEventWatcher] INFO: Scout notices: [{"level": "DEBUG", "message": "Returning cached result"}]
ACCESS [2019-03-18T17:58:34.491Z] "GET /httpbin/ HTTP/1.1" 404 NR 0 0 0 - "<IP_ADDRESS>" "curl/7.29.0" "c789a46d-02b0-42ab-b946-a78f0c812dcb" "<IP_ADDRESS>" "-"
ACCESS [2019-03-18T17:58:37.508Z] "GET /httpbin/ HTTP/1.1" 200 - 0 9593 29 28 "<IP_ADDRESS>" "curl/7.29.0" "df9305cb-ec0e-43fc-ad53-40911507cbb4" "httpbin.org" "<IP_ADDRESS>:80"

How many service objects exist across all namespaces in your cluster? I wonder if this is related to https://github.com/datawire/ambassador/issues/1292

We have close to 800 and it will increase in future.

This will, I hope, be addressed by 0.52.0's switch to the C YAML parser.

The validation is fast now, but the envoy loading is still slow. Below is the logs. If i have the service annotation at 2019/03/21 20:33:30, my first success response is ACCESS [2019-03-21T20:34:13.335Z] ".
It took almost 45 seconds to see my update.
2019/03/21 20:33:30 SYNC: python3 /ambassador/post_update.py http://localhost:35512/api/snapshot/4
2019-03-21 20:33:30 diagd 0.52.0 [P51TThreadPoolExecutor-0_15] INFO: Update requested from http://localhost:35512/api/snapshot/4
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: copying configuration from http://localhost:35512/api/snapshot/4 to /ambassador/snapshots/snapshot-tmp.yaml
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://dev-key-value-service.services:8080 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://employee-rs-service.services:8080 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://httpbin.org:80 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://key-value-service.services:8080 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://kube-docs.od-system:8080 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://<IP_ADDRESS>:8877 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://<IP_ADDRESS>:8877 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: Load balancer for tcp://<IP_ADDRESS>:8877 is None
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: listen_ports ['80']
2019-03-21 20:33:30 diagd 0.52.0 [P51TAmbassadorEventWatcher] INFO: -global-: NOTICE: Ambassador 0.60 will default to listening on port 8080 for HTTP.  It shows that multiple people are seeing it.

@songole I'm not sure whats the original issue #. I can link.

Lots of work around this in 0.60.3. I'm going to close this (again) -- please reopen (again) if there are issues on 0.61!


Auto-complete for "git commit --cleanup=" doesn't provide option values

The Git auto-complete provider (Git.ts) appears to provide the 5 possible "cleanup" mode values for the command "git commit --cleanup=" i.e. modes "strip", "whitespace", "verbatim", "scissors" "or default".
See code at https://github.com/shockone/black-screen/blob/master/src/plugins/autocompletion_providers/Git.ts#l206.
When I type "git commit --cleanup=" in a directory with changes ready to commit (staged) to a git repository the 5 "cleanup" mode options aren't suggested in the auto-complete box. Is this expected behavior.
See screenshot attached.

I'm experimenting with updates to the grep auto-complete provider (Grep.ts) and want to add the same functionality for a grep argument which has a fixed set of options.

@mkenne11 autocompletion will be heavily rewritten. Please hold on with your changes unless you're willing to rewrite them as well.

Thanks I'll hold off to the autocomplete rewrite :)

Fixed.


Improve experience when manually updating extension

Testing #39593
When manually updating an extension, you start VSCode and have to wait 10 seconds to get a  dialog that asks to reload until you see your extension.
I often test my extensions like that, it would be great if the experience could be improved:

reduce the timeout
place the cache file in the extension folder, with a good name (extensions.cache) so I can delete it myself if needed
check the folder mtime as suggested in #39652

The timeout is set to 5 seconds. I will not reduce it further as to not risk overlapping with the startup sequence.
Let's track the mtime in #39652

==> I suggest we leave this issue about moving the cache file to the user extensions folder.

I haven't received any other complaints about the cache location. I will close this for now, please speak up if you consider it to be super important...


Minor changes

Thanks for the patch! This looks good!

@blowekamp Do I need to send this to gerrit next?

Topic has been merged into next.


Add contributing section in docs

The current documentation in Read the Docs doesn't contain the information about contributing present in CONTRIBUTING.rst
In order to solve this, knowledge about how to write documentation in Read the Docs is required.

Hi can I take this?

@michiboo, this was closed in #640!


Fixed #483 Import Data Into Ballot Item table

Aside from the comment I added, this is ready to go! 👍


0.6 doesn't work on windows

So, I tried to update this module because I had been using 0.5.0 for the longest time, but this broke everything on windows for me.
My code:
require('gaze')(['metadata.js', 'src/**/*', 'lib/**/*', 'Slakefile'])(function () {
  console.log('Watching files...');
  this.on('all', console.log.bind(console));
});

This prints "Watching files..." as many times as there are files, then nothing is printed when I try to trigger an event..?

Which version of Windows and Node.js or io.js?
Also this part looks off:
require('gaze')(['metadata.js', 'src/**/*', 'lib/**/*', 'Slakefile'])(function () {

I think it should be:
require('gaze')(['metadata.js', 'src/**/*', 'lib/**/*', 'Slakefile'], function () {

yes, sorry, that was a translation mistake. Windows 8.1, node 0.12.4


Document cgroup v1 issue for Docker desktop on Mac since v4.3.0

The cgroup v1 issue was discussed in this thread before: https://github.com/srl-labs/containerlab/pull/470
Nevertheless, I'd suggest to add a note specifically for containerlab on MacOS to inform the user about changes in v4.3.0 and v4.4.2: https://docs.docker.com/desktop/mac/release-notes/
One reason to do that is the problem with docker desktop GUI and the deprecatedCgroupv1 flag. When configured in Docker desktop preferences, it may not work or crash Docker desktop.
The working solution is documented here:
https://github.com/docker/for-mac/issues/6073
I'd suggest to refer to this thread or add required commands directly to the Containerlab guide:
# Stop running Docker
test -z "$(docker ps -q 2>/dev/null)" && osascript -e 'quit app "Docker"'
# Install jq and moreutils so we can merge into the existing json file
brew install jq moreutils
# Add the needed cgroup config to docker settings.json
echo '{"deprecatedCgroupv1": true}' | \
  jq -s '.[0] * .[1]' ~/Library/Group\ Containers/group.com.docker/settings.json - | \
  sponge ~/Library/Group\ Containers/group.com.docker/settings.json
# Restart docker desktop
open --background -a Docker

Hi @ankudinov
so this is only needed for those images which still require cgroupv1, like ceos on mac?


Mc events

@birdage @DanielJMaher This is ready for review, going to get it pretty next week.

@birdage


Update the aks install/remove connector to use virtual-kubelet

This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

Command Guidelines

[x] Each command and parameter has a meaningful description.
[ ] Each new command has a test.

(see Authoring Command Modules)

View a preview at https://prompt.ws/r/Azure/azure-cli/5091
This is an experimental preview for @microsoft.com users.
(It may take a minute or two for your instance to be ready)
Email feedback to 'azfeedback' with subject 'Prompt Feedback'.

thanks for the feedbacks. address them and fix all pylint and flake8 violations


Fix path construction bug in shell e2e

@roberthbailey I don't have the bandwidth to rewrite this test like we discussed, but I can at least fix the path construction bug.

Shippable is complaining (legitimately) that this leaves unused imports which break the build. Can you fix those?

Weak, my bad.  I had thought this built.

I know what I did now.  I took logic for skips if the err was os.IsNotExist.

LGTM.


Maxcdn has shut down, cdn not working anymore.

Is there an alternative cdn? Max cdn has shut down and https://twemoji.maxcdn.com/v/latest/twemoji.min.js is down as well.

@WebReflection amend
User-land Solution
https://github.com/twitter/twemoji/issues/580#issuecomment-1376299586
twemoji.parse(
  document.body,
  { base<EMAIL_ADDRESS>}
)

What is going on?
MaxCDN doesn't exist anymore and we had it hard coded as default option but that was never meant to be the only CDN able to serve twemoji so that you can simply specify a base field as option to parse with a different CDN or wait for the new package to be deployed on npm (📣 the new package is called @twemoji/api and has been already deployed) and use the new hard coded default. The lesson to learn here is that some CDN might not be there forever, so it's always good to know there are escape hatches within the code, like it is for twemoji.
📣 New Package Details
https://github.com/twitter/twemoji/issues/580#issuecomment-1380933650

via GIPHY

there is no js file there though, only images..

Here is an alternative CDN for twemojis: https://cdnjs.cloudflare.com/ajax/libs/twemoji/14.0.2/72x72/1f1e6-1f1fd.png

True that

I'm gonna make a PR. The images doesn't need to be on the same CDN. If<EMAIL_ADDRESS>uses cloudflare url.

Get the gh-pages branch
Go to gh-pages branch -> 12.0.4
Move the twemoji.min.js file, 72X72 & svg folders wherever you'd like them
Update the code at the beginning of twemoji.min.js and change the base to point where you put the 72X72 & svg folders
var twemoji=function(){"use strict";var twemoji={base:"/mynewlocation/"

As a quick fix i took twemoji-parser and manually changed the index.js code where it directs to max.cdn to the new cdn.
This is a very fast fix if your code is in production

I can assure you my code hasn't changed in 7 years so if you keep license around, as you should, and just update the static parts to point at a different path, everything should work as expected.
the related PR already addressed this, so fork away and live happily ever after.

@simplexx among all the things, twemoji has always been a Twitter service for the community.
At my times in there, we had a great agreement / deal with MaxCDN so that it's hard to blame the boss this time, as MaxCDN is a completely different company/story.
What I see is some poor attention to this project, as companies don't close from a day to another (usually?) but as we all know what's going on @ twitter, I can't really blame any of my former colleagues, or new arrivals there.
Please let's not make it a wall of shame for all the people that worked at this, thanks for your understanding (I've left 7 years ago or more, as example, I've got pinged and trying to help anyway).

For what it's worth, while I was still there we were in talks with MaxCDN to have the same deal when they migrated to be Stackpath. Everyone who had worked on the deal with MaxCDN had left and left no record of that deal, so it was taking them a bit longer to work out than expected – MaxCDN used to offer free hosting to OSS projects, but Stackpath wouldn't be doing that. They were working on an exception for us, but any emails they send to our Twitter emails now get bounced, so I guess they could've gotten this sorted out before shutting down MaxCDN... but it's not like we're there on the other end to make it happen, so it appears we'll never know.

Yeah, we didn't exactly get a chance to hand anything over before getting laid off. Usually a company with the ability to plan business operations would allot time for that, but we all got spontaneously locked out of our computers in the middle of the night, so. Poor handover would imply having tried to hand over. 🥲
The regex comes from https://github.com/twitter/twemoji-parser, where I open sourced the Scala generator for the regex last year so we aren't missing anything there. It also has the same problem of the default options being broken due to MaxCDN, but specifying another path is possible as an option. Assets are kept only in twemoji not twemoji-parser. For longer term maintenance beyond just this one issue, we'd eventually need to fork twemoji-parser too for regex updates – I have NPM publish access to it, but there actually seems to be nobody left with GitHub write access who previously committed to it.
I'd be pretty sad to deprecate Twemoji honestly, especially when the designer is willing to work on a fork of it with me. I was hoping to get wherever I worked next to adopt Twemoji though, so the details are a little up in the air. Maybe we just fork now for compatibility but without substantial support, although I know that would annoy some people.

@WebReflection @jdecked I had posted in the PR but will add it here. My business partner and I discussed and our company would be potentially interested in keeping it going either on a fork or a completely new repository. We using it a ton at our company and replacing it would actually be more of a pain than maintaining/improving it.

Yosh,
First things first: sorry for the annoyance and ten thousands thanks to ex-employees for the fast reaction.
For me Twemoji is part of the internet furniture and I definitely would like to see it continuing it's existence.
I hope #581 can end up merged and deployed to put ducktape on the leak, but I don't consider this as a long-term solution and it will be a pleasure to help if a fork emerges.
@jdecked I definitely think it's better to fork even without substantial support to ensure compatibility and let the door of "yeah today I have some time let review a PR or fix an issue" open.
Having people that already worked on it and even the designer would give a realy chance to the fork to live and having people that want to see it live like @burgoyn1 or me could discharge the "owners" of some of the annoying work.

If you don't want to use a fork, this was my hot fix:
result = twemoji.parse(segment.text, {
  folder: 'svg',
  ext: '.svg',
});
// temporary fix because maxcdn has stopped supporting twemoji
return result.replace(
  'twemoji.maxcdn.com/v',
  'cdnjs.cloudflare.com/ajax/libs/twemoji',
);

Consider switching to jsDelivr https://www.jsdelivr.com/package/npm/twemoji
We also host Bootstrap CDN.

Consider switching to jsDelivr https://www.jsdelivr.com/package/npm/twemoji We also host Bootstrap CDN.

The problem is the images/svg still point to maxcdn.com. Those images are not on jsdeliver.

If someone who controls the domain is reading then please get in contact cause we can provide a custom CDN to get all sites back online

@jimaek after pointing to https://cdnjs.cloudflare.com/ajax/libs/emojione/2.2.7/assets/svg/.svg emojis are loading fine, do you have any other custom CDN path?

FYI how to use jsDelivr to serve twemoji images:
twemoji.parse(
  document.body,
  { base<EMAIL_ADDRESS>}
)

@jdecked I believe what's next, as I've merged into master the CDN change, is to actually run npm run deploy or basically bring the dist folder into gh-pages branch, and then publish from there to npm ... this should just bring in latest .js files with the different CDN, and upload the new package that should work out of the box. Dare I say, as this is currently fully broken for everyone, a patch version might be what everyone would hope for, but then again, history told us sneaky patches are bad, so at least a minor, if not a major, would be needed, imho ... but I let you decide on that.

@yhatt yes, the library is meant to run even on localhost, so that changing the base field should always work (it does on my machine 😅) however, the default is a broken CDN, and I think this should be fixed.
I don't think I can deploy to npm but if @jdecked somehow has issues in porting master to gh-pages branch and publish from it, which is basically what deploy.sh does, I can try and eventually fail at publishing time.

For everyone else looking for the quick and dirty summary fix:
<script<EMAIL_ADDRESS>crossorigin="anonymous"></script>
<script>
  twemoji.parse(document.body, { base<EMAIL_ADDRESS>});
 </script>

However, this is now relying on two CDNs staying up.    Possible to use just one?

unpkg only hosts JavaScript files (https://unpkg.com/browse/twemoji@14.0.2/), and cdnjs only hosts emoji images (https://cdnjs.com/libraries/twemoji). jsDelivr can host both, by using npm entrypoint and github entrypoint.
<EMAIL_ADDRESS>

unpkg only hosts JavaScript files (https://unpkg.com/browse/twemoji@14.0.2/), and cdnjs only hosts emoji images (https://cdnjs.com/libraries/twemoji). jsDelivr can host both, by using npm entrypoint and github entrypoint.
<EMAIL_ADDRESS>
Thank you!  that's the URL I was looking for.  https://github.com/twitter/twemoji/issues/580#issuecomment-1376450456 updated.

Hi @WebReflection,
Dose this patch fix also build older version of twemoji? (eg. version 2 in<EMAIL_ADDRESS>

Dose this patch fix also build older version of twemoji?

@tkforce nothing can replace semver packages and nobody is going to backport changes ... like I've said, the JS code is the same since 7 years ago so any version that worked before, will work when you switch to latest or just use a different base configuration ... don't expect 2.2.2 to be magically fixed, it can't happen by semver design.

If you don't want to use a fork, this was my hot fix in the one place we used twemoji:
result = twemoji.parse(segment.text, {
  folder: 'svg',
  ext: '.svg',
});
// temporary fix because maxcdn has stopped supporting twemoji
return result.replace(
  'twemoji.maxcdn.com/v',
  'cdnjs.cloudflare.com/ajax/libs/twemoji',
);

Works like a charm.

Does anyone know why this has stopped working?

quick look at the code shows that this url is configurable,
try pass parameter 'base' in options(second parameter) for parse method
for example
result = twemoji.parse(segment.text, {
  folder: 'svg',
  ext: '.svg',
  **base: 'https://cdnjs.cloudflare.com/ajax/libs/twemoji/14.0.1/'**
});

should work
(works for me with react-twemoji, which uses twemoji 14.0.1

Hello World
the fix you are all suggesting is in the README and it's been there since day zero.
There is a base option documented (probably not superbly) which goal is to let you define any CDN you like as URL for emoji.
I've already said that's the right way to go but apparently there's some issue in communicating that better than my tweet, this issue itself, or the README.
Maybe somebody should edit the top post to point at the base solution, as that was meant to be working since day zero in here, but apparently many never noticed the feature?

The problem with the base solution is that if you are using typescript, it will not accept base in the options, I believe.

@karptonite 7 years ago TypeScript didn't even exist ... not sure I should apologize for using what the standard HTML knows as base URL since long time ago ... or maybe acknowledge TS should "sometimes" mind his business? It produces JS and this module is, and has always been, JS.

And base basically works with TypeScript as the .d.ts file mentions it in the options: https://github.com/twitter/twemoji/blob/master/index.d.ts#L18
I don't know why it would fail with TypeScript, but if it does then it's your problem and not Twemoji's or WebReflection's problem.

Twemoji is by far my favorite emoji style, and I never really thought about the fact that everything going on @ Twitter might affect that, I really hope this gets resolved and it can continue on into the future. If twemoji ever gets scrapped I'll be very disappointed, but I have hope that the community would figure out a way to keep it going.

@Clari-7744 it's already resolved, it has always been https://github.com/twitter/twemoji/issues/580#issuecomment-1376299586 ... all you need to do is to change where you get assets from, as mentioned in that post.
The npm package has the library like always, the parse needs to point at a different CDN.
Due legal constraints and the fact only ex employees are trying to fix this, it's very likely twemoji will be published on npm with a different name, so nothing looks like will be fixed in here but there is an API field that helps.

The problem with the base solution is that if you are using typescript, it will not accept base in the options, I believe.

Other replies to this are right--the problem was in my own typescript definitions.

I share with you an alternative to the Twemoji library in less than 10 lines
const emojiFinder = /(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000-\udfff]|\ud83e[\ud000-\udfff])/g;

export default function parse(html: string) {
  return html.replace(emojiFinder, (match) =>
    `<img alt="${match}" src="https://abs-0.twimg.com/emoji/v2/svg/${
      (match.codePointAt(0) ?? 0).toString(16)
    }.svg">`,
  );
}

@Mathieu2301 that’s not how anyone should fix the default callback issue. Please don’t recommend unnecessary home made solutions, thanks.

Alright, we're almost there: the last piece is that it seems that @unvented created an NPM organization by the name of "twemoji" already, so I can't publish under the @twemoji namespace. It seems like the organization has only one package, last published over a year ago, and that future publishes for that package went under the "twemazing" org.
Samuel – would you mind giving up the twemoji org on NPM to keep things going here? :pray:

@jdecked I'll assign it to you once i'm at home.

Alright, we're almost there: the last piece is that it seems that @unvented created an NPM organization by the name of "twemoji" already, so I can't publish under the @twemoji namespace. It seems like the organization has only one package, last published over a year ago, and that future publishes for that package went under the "twemazing" org.
Samuel – would you mind giving up the twemoji org on NPM to keep things going here? 🙏

This project seems to be dead (last commit in April 2022) and probably no new features will be added in the future. I thought all the core maintainers of Twemoji were laid off, so why exactly do you need the organization?
I would love to hand it over if Twemoji was actively maintained again, but I don't see a reason for a transfer right now, sorry.
Also, as a suggestion regarding this issue:
You should give jsdelivr a try, they tend to have better loading times compared to unpkg.
~ Cheers

@jdecked also, why give Twitter the pleasure by naming the fork after it? Give the library a new name and make it a standalone open source project!

This project seems to be dead (last commit in April 2022) and probably no new features will be added in the future. I thought all the core maintainers of Twemoji were laid off, so why exactly do you need the organization?
I would love to hand it over if Twemoji was actively maintained again, but I don't see a reason for a transfer right now, sorry.

That's exactly it: the old maintainers of Twemoji are making a fork so it is actively maintained again, because we don't want to see it die. We don't want to publish to the original Twemoji package on NPM for legal reasons (we could theoretically get in trouble even if we have the right permissions). The organization would be used to house the fork for the community, not in a corporate way for Twitter or anything.

Also, as a suggestion regarding this issue:
You should give jsdelivr a try, they tend to have better loading times compared to unpkg/cdnjs.

This is exactly what we're doing in the fork, which is currently waiting at https://github.com/jdecked/twemoji to be published.
Why not call the fork something different? Twemoji isn't some corporate name developed by a marketing team. It came from the people who worked on it, so that's why it has special meaning. Also, the style will be the same, so giving it the same name makes it easier to find for the sake of continuity.

@jdecked I invited you to the org, please accept the invite.

@jdecked If you need any help or want any additional maintainers my company is willing to put resources behind it to keep it open sourced and update it (as I stated above). We use it a lot on our product and would be more than happy to contribute to help improve it.

@unvented thanks, and yes. you can reach me on Discord as jdecked#3434 or on Mastodon as<EMAIL_ADDRESS>

@unvented and I connected over Discord and we're all set now. Thanks again! I've deleted the comment with my contact info to minimize unsolicited messages.

Thanks @unvented and thanks @jdecked for the follow up 🍻

P.S. features here come from Unicode standard ... they announce new emoji, the emoji-parser gets its RegExp updated among assets. The JS library though, worked well "forever" either as DOM library or as HTML parser, but it as born in days ShadowDOM really didn't exist so that ShadowDOM, when a .shadowRoot is available, or as tweak/hack around the Element.prototype.attachShadow is all it's needed to have twemoji up with the 2023 DOM specs.
This is to say that the JS part has never been more battle tested and reliable than it is now already, but maintenance, like it is for any software out there due changes or updates around their scope, is needed ... and I hope twemoji will be kept alive to satisfy these new, or old, needs.

I've published an official fork of Twemoji at @twemoji/api on NPM, starting at v14.1.0. The default CDN is jsDelivr, so this issue (#580) is resolved. The code for the fork is hosted on GitHub here. Similarly, an official fork of twemoji-parser is available at @twemoji/parser on NPM and that code is hosted on GitHub here.
Feel free to keep using Twitter's twemoji and twemoji-parser, but I don't have any more planned updates to it.

@jdecked awesome, and thanks for the invitation 🙏 ... I think we can close this after maybe amending the OP about this?

@WebReflection yep, sounds good to me!

This issue has been fixed through a fork and the new module is called @twemoji/api and it's already available on npm.
I am closing this issue as no more work will be done in this repository around this library, or better ... nobody that doesn't work @ Twitter would likely do anything in here to change the current state.
Use an explicit base options' field, as specified in the README, if you don't want to switch to the current fork, or just switch to the fork which is being maintained by people that care about this project 👋

@jdecked do we need to also add the definitelyTyped files for the new @twemoji/parser lib? And also shouldn't we update this naming to match the new module name? https://github.com/jdecked/twemoji/blob/main/index.d.ts#L93

@jdecked do we need to also add the definitelyTyped files for the new @twemoji/parser lib? And also shouldn't we update this naming to match the new module name? https://github.com/jdecked/twemoji/blob/main/index.d.ts#L93
Otherwise we don't get typescript support I guess?

Twemoji Parser has never had native TypeScript support. Feel free to PR types if you want them; I'm largely uninterested in maintaining types for either of these libraries given I'm not a TS user, but just as before I'll continue to accept PRs contributing TS types from people who want them.
Good catch on the naming for the Twemoji TS module. Since it worked before, I'll update the naming next week as I'm currently travelling – or someone is free to PR, which I can merge from my phone. :-)

Good catch on the naming for the Twemoji TS module. Since it worked before, I'll update the naming next week as I'm currently travelling – or someone is free to PR, which I can merge from my phone. :-)

I tried the fix & in theory it works. 😅
If I try apply this change to the actually shipped NPM package (to check it actually works as expected), I have errors. 😕
What I don't really get is: why is the index.d.ts file that's shipped with the new fork different from the previous one?

I've created a first pull request https://github.com/jdecked/twemoji/pull/1 to improve TypeScript compabillity. Cheers 🍻

I had the same issue with my react app but after updating the react-easy-emoji to version ^1.8.1 that resolved the issue.
Apparently they resolved the issue in the last patch

The CDN is back online! All previous URLs will continue to work and redirect to jsDelivr
e.g. https://twemoji.maxcdn.com/2/svg/1f525.svg
Stackpath reached out to @jsDelivr and changed the DNS CNAME to point to a CDN we control.

Looks like CORS isn't working correctly, so the script https://twemoji.maxcdn.com/v/latest/twemoji.min.js still doesn't work

Are you using a cached version by some intermediate proxy? Because that URL redirects to<EMAIL_ADDRESS>which has all the CORS header in place.
It seem like your device is not following the redirect for some reason

@jimaek it seems that browsers don't follow redirects when CORS is required but the corresponding headers are missing

Can you try again please? I think they should be present now

Consider switching to jsDelivr https://www.jsdelivr.com/package/npm/twemoji We also host Bootstrap CDN. -@jimakek

The problem is the images/svg still point to maxcdn.com. Those images are not on jsdeliver.

Yikes, I came here just to say JSDeliver did not cache the images as well that were pointed another place.
@jimaek - sorry but your efforts were appreciated!

If you use the native files from the CDN and not the redirected it should work properly. If you encountered issues please let me know.

Drop-in replacement for Twemoji: https://github.com/AdvenaHQ/fluent-emoji

Twemoji seems to be broken yet again.

Yes, I ended up self-hosting it.

Please consider moving to jsDelivr directly https://www.jsdelivr.com/package/npm/twemoji It's a multi-CDN with focus on production use.
https://cdn.rawgit.com has temporarily issues due to a BunnyCDN outage

The problem is that the assets are down (the actual emoji images), which causes issues with rendering and blocks the site loading. I suggest using this: https://github.com/jdecked/twemoji - I tested it and it works perfectly well. Also, @jdecked, who worked on the original script, is actively developing it! In my opinion this project here should be put to rest and the first sentence of the description should link to https://github.com/jdecked/twemoji. This would be the best solution for everybody.

In the meantime, this is happening: From Bitdefender:
chrome.exe attempted to establish a connection relying on an unmatching security certificate to twemoji.maxcdn.com. We blocked the connection to keep your data safe since the used certificate was issued for a different web address than the targeted one.

Stackpath broke the CDN by changing their DNS and pointing it to a different IP. In the past they worked with us to ensure nothing breaks, but this time they simply made the switch without warning. Since the *.maxcdn.com domain is controlled by them there is nothing we can do to fix it unfortunately. I recommend to simply switch to jsDelivr, which is a CDN focused on production use cases

@jimaek jsDelivr could host a twemoji.jsdelivr.net since currently there is no alternative on cdn.jsdelivr.net:
<EMAIL_ADDRESS>
The assets are still point to twemoji.maxcdn.com and those assets are not on cdn.jsdelivr.net.

as first creator and maintainer of this project, where literally nothing changed since 2014 around my written code, I would be happy to help migrating the CDN from max to jsdelivr if that's all it takes to have this project still up and running.


Can be replaced with ResponseStatusException

More convenient way is RSE. It would be automatically handled by Spring
https://github.com/tracehubpm/pmo/blob/6ba5195fb667ab6beb67af340942bcd67699c469/src/main/java/git/tracehub/pmo/exception/ResourceAlreadyExistsException.java#L32
https://github.com/tracehubpm/pmo/blob/6ba5195fb667ab6beb67af340942bcd67699c469/src/main/java/git/tracehub/pmo/exception/ResourceNotFoundException.java#L32
Can be replaced with
public class SomeExcpetion extends ResponseStatusException {

Response:
HTTP/1.1 404
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Sat, 26 Dec 2020 19:38:09 GMT

{
    "timestamp": "2020-12-26T19:38:09.426+00:00",
    "status": 404,
    "error": "Not Found",
    "path": "/some/path"
}

@l3r8yJ thanks for bug report, @hizmailovich take a look, please


[#1648] Failed to fetch UTXOs: code 14

Closes #1648
The SDK, after our latest changes, reports an error from its fetchUtxos API to clients, but it omits to distinguish between the loss of connection state and the others. It's fixed with this PR.
This also impacts SDK’s ability to recover from loss of connection state later
Changelog updated

Note
This code review checklist is intended to serve as a starting point for the author and reviewer, although it may not be appropriate for all types of changes (e.g. fixing a spelling typo in documentation).  For more in-depth discussion of how we think about code review, please see Code Review Guidelines.

Author

[x] Self-review your own code in GitHub's web interface[^1]
[ ] Add automated tests as appropriate
[ ] Update the manual tests[^2] as appropriate
[ ] Check the code coverage[^3] report for the automated tests
[x] Update documentation as appropriate (e.g README.md, Architecture.md, etc.)
[x] Run the demo app and try the changes
[x] Pull in the latest changes from the main branch and squash your commits before assigning a reviewer[^4]

Reviewer

[ ] Check the code with the Code Review Guidelines checklist
[ ] Perform an ad hoc review[^5]
[ ] Review the automated tests
[ ] Review the manual tests
[ ] Review the documentation, README.md, Architecture.md, etc. as appropriate
[ ] Run the demo app and try the changes[^6]

[^1]: Code often looks different when reviewing the diff in a browser, making it easier to spot potential bugs.
[^2]: While we aim for automated testing of the SDK, some aspects require manual testing. If you had to manually test
something during development of this pull request, write those steps down.
[^3]: While we are not looking for perfect coverage, the tool can point out potential cases that have been missed. Code coverage can be generated with: ./gradlew check for Kotlin modules and ./gradlew connectedCheck -PIS_ANDROID_INSTRUMENTATION_TEST_COVERAGE_ENABLED=true for Android modules.
[^4]: Having your code up to date and squashed will make it easier for others to review. Use best judgement when squashing commits, as some changes (such as refactoring) might be easier to review as a separate commit.
[^5]: In addition to a first pass using the code review guidelines, do a second pass using your best judgement and experience which may identify additional questions or comments. Research shows that code review is most effective when done in multiple passes, where reviewers look for different things through each pass.
[^6]: While the CI server runs the demo app to look for build failures or crashes, humans running the demo app are
more likely to notice unexpected log messages, UI inconsistencies, or bad output data. Perform this step last, after verifying the code changes are safe to run locally.

Fyi, the failing tests are part of the feature branch and will be resolved separately.


Review/remove the guide about actors

The guide: https://github.com/hyperledger/iroha/blob/iroha2-dev/docs/source/guides/actor.md

We should probably remove this internal doc. It doesn't quite belong in this repository, and very unlikely to be touched by someone who is qualified to actually review the documentation in this state.

see https://github.com/hyperledger/iroha/pull/2872/files#r998384143

Removed in https://github.com/hyperledger/iroha/pull/2895


Refactor.home page pom

Slightly updates homePage POM with data-testIds and removes unnecessary okButton locator.
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] I have added tests wherever relevant
[x] I have made sure that all automated checks pass before requesting a review
[x] I do not need any deviation from our PR guidelines

I made some changes in the names of testids. I hope they are a bit better now. Thanks for pointing this out @aberonni

A lot of the test ids do not mention the component name in the id itself, which is a guideline I think we established in a previous PR, and which would make it easier to make sure that we don't have duplicate test ids. Could you make sure you check the new ids for this? The rest seems good :)


Contrib project

I finally started with a contrib-project! The repo is under my account cwe1ss/opentracing-contrib-dotnet but it would be great if we could move this to opentracing-contrib at some point.
Right now, it traces incoming HTTP requests in ASP.NET Core and outgoing HTTP requests with HttpClient.
It's not tested because there's no tracer implementation yet :smile:. Hopefully, I'll find some time over the next few weeks to work on BasicTracer and get a working integration with e.g. Zipkin. I can't wait to finally see a first trace in a GUI :smile:.

@cwe1ss Nice work.
What do you think of creating a separate brand to host all these extra parts, together with all extension methods we removed from OT to keep API surface small. The list of surrounding projects will grow and I think it is not necessarily a bad thing to create a separate brand than opentracing around them. I think that it could both be an OSS implementation of the OpenTracing API, together with this type of adaptors. We would then be more flexible then using the open tracing in everything.
I came up with the name MicroCrumbs a while ago where I planed to put this kind of code. If you like it we can use it or if you have a better name I'm open for that as well.

thanks - still a lot to do though! :smile:
There's the separate github organization opentracing-contrib which is meant for language-specific projects. Being listed there would probably give us the best "marketing push" because the sad truth is, no-one knows about this in the .NET world yet. :(
My concerns with a separate name are that it will be harder for people to understand the relationship. By using OpenTracing.Contrib, we would share the name and we should pop up whenever someone is searching for opentracing and .NET.
Also, the implementations are not really projects themselves - they just add instrumentation for existing libraries/frameworks. Having OpenTracing.Contrib.AspNetCore (or just OpenTracing.AspNetCore if we don't need the Contrib tag) would be easier to understand than MicroCrumbs.AspNetCore.
MicroCrumbs sounds great for a tracing system though! If you plan to build a .NET based tracer, I'd definitely vote for it :)
Your thoughts?

/cc @adriancole @bensigelman @dawallin @yurishkuro
I now have a first working "alpha" version of the following .NET / C# packages. Before we take a closer look at them and move them to opentracing-contrib, I'd like to get your feedback regarding the package names and some general things!

OpenTracing.Instrumentation: Targets library & application developers. It contains in-process trace propagation, helper methods and instrumentation for HTTP calls and database calls (done by an ORM tool). Thanks to some nice hooks in .NET, this works without package dependencies, so I included it in this package.
OpenTracing.Instrumentation.AspNetCore: Instrumentation for the "ASP.NET Core" web framework (incoming requests, some MVC performance). Depends on OpenTracing.Instrumentation.
OpenTracing.Testing: In-memory tracer for unit tests.
OpenTracing.Tracer.Abstractions: Contains base classes for tracers. Similar to the Java AbstractTracer. Ideally, this would also be used by the BasicTracer.
OpenTracing.Tracer.BatchReporter: Reusable library for tracers that queues spans and allows tracers to send spans in batches on a background thread.
OpenTracing.Tracer.Zipkin: A tracer that sends data to Zipkin. This currently is a mix of some existing Zipkin libraries and Uber's jaeger.

My main questions:

Should the packages include the Contrib part? I don't have it right now because to me it seems more readable without it (e.g. OpenTracing.Instrumentation.AspNetCore vs. OpenTracing.Contrib.Instrumentation.AspNetCore)
Should the tracer-packages be called OpenTracing.Tracer.*? There's the go-tracer "Tracer" so this might be problematic?! (especially if we have a OpenTracing.Tracer.Tracer package one day). PS: I don't think "tracer" is a good product name for a "tracer" :)
Should tracer libraries be in the opentracing-contrib repo or should they be in the tracer repos? @adriancole Are you interested in hosting this package in a github.com/openzipkin repo with me as an owner?
Should OpenTracing.Tracer.Abstractions be moved into the opentracing org? Java's AbstractTracer is there as well. Also, it would make sense to use this package for the BasicTracer implementation which currently is in the opentracing org as well.

Thanks for your time and feedback!

wrt the "zipkin tracer" thing (combination of inject/extract and reporting I suspect?).. initially it is probably easiest to keep it here as that would eliminate some version drift as the thing evolves.
There are some incompatibility issues (like the recent dict+timestamp logging api), which require common tactics that should be applied similarly across zipkin implementations. Basically, success occurs when folks watching the code have equal stake in OpenTracing and Zipkin, as for example, things like dict logging are a mess otherwise (people dumping junk in zipkin as they don't care since they use something else).
Long story short, the best example OpenTracing and Zipkin working is zipkin-opentracing-go, led by @basvanbeek. If we had something similar in openzipkin for csharp/.net, it would have the same chance of success, not just now, but as apis continue to drift. If you are interested in championing similarly and/or inviting others in zipkin and/or bas, then I'd be happy to make a repo in openzipkin for this, and give you whatever access you need.

Should the packages include the Contrib part? I don't have it right now because to me it seems more readable without it (e.g. OpenTracing.Instrumentation.AspNetCore vs. OpenTracing.Contrib.Instrumentation.AspNetCore)

I have a slight pref for including contrib in the artifact names, but that's just me. It will make it clearer (in some future scenario) which (few) artifacts are "core".

Should the tracer-packages be called OpenTracing.Tracer.*? There's the go-tracer "Tracer" so this might be problematic?! (especially if we have a OpenTracing.Tracer.Tracer package one day). PS: I don't think "tracer" is a good product name for a "tracer" :)

They're just there as helpers, right? Maybe (???) OpenTracing.TracerUtil.*? Just a thought. What you have now is also fine with me.
(And re your PS: I agree!)

Should tracer libraries be in the opentracing-contrib repo or should they be in the tracer repos? @adriancole Are you interested in hosting this package in a github.com/openzipkin repo with me as an owner?

I think it's more sensible to have an OT-Zipkin Tracer live in a zipkin repo, but that's ultimately up to Zipkin (and Adrian has weighed in, of course).

Should OpenTracing.Tracer.Abstractions be moved into the opentracing org? Java's AbstractTracer is there as well. Also, it would make sense to use this package for the BasicTracer implementation which currently is in the opentracing org as well.

I really dislike where AbstractTracer ended up in Java, so I'd rather have them in contrib, at least for now. Having them in the core git repo implies a level of stability I don't think is appropriate.
Thanks! (And sorry for the delay)

@cwe1ss 👍

I moved my instrumentation code for ASP.NET Core into the new opentracing-contrib/csharp-aspnetcore project. I'd like to invite everyone who's interested to contribute there as well!
I'm especially interested in feedback on
https://github.com/opentracing-contrib/csharp-aspnetcore/issues/2 which discusses the package/library names for nuget.org


renaming folders for exercises

take a look at the folders for rma exercises, as this section comes after nonblocking I chose a different name starting from 1X_***** @robertodr

Hi Pedro, can you please run:
$ git fetch origin
$ git rebase -i origin/master

to rebase your branch? It's hard to follow the diff in the web interface otherwise.


[136172] Added rowsPerPageTitle to customize rows per page text

Added rowsPerPageTitle to customize rows per page text for pagination footer. Fixes 136172
Fixes : https://github.com/flutter/flutter/issues/136172
Added  new parameter rowsPerPageTitle in order to customize the Rows Per Page text for Pagination in PaginatedDataTableClass
*Screenshots: *
Before:

After:

Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Added unit test for the new Parameter. @TahaTesser

Thanks for the suggestions @TahaTesser.
Updated branch with your suggestions and unit tests passing locally 🚀

Hey @Piinks requesting your review here :)

Closing as we might want to use Localization to achieve this