Brace stripping for diacritics

I know it is a painful subject, but I'd like to bring it up for hopefully one last time (see also https://github.com/plk/biber/issues/297#issuecomment-583756230). Especially since we now have expl3 case change in biblatex (https://github.com/plk/biblatex/pull/1005).
Biber already correctly strips the outer braces in constructs such as
{\'I} -- {\v C}

so that they appear in the .bbl only as
Í -- Č

But currently the additional inner/argument braces in the equivalent version
{\' {I}} -- {\v{C}}

are not stripped, leaving us with
{Í} -- {Č}

This has negative effects for case protection. With classical BibTeX both forms are not case protected, but since Biber does not strip the braces, Biber will accidentally brace protect the latter form.
Compare
\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{csquotes}

\begin{filecontents}[force]{\jobname.bib}
@article{lorem:a,
  title   = {ALorem and {{\v C}esk\'a} republika},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:b,
  title   = {BLorem and {{\v{C}}esk\'{a}} republikb},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:c,
  title   = {CLorem and {\v C}esk\'a republikc},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:d,
  title   = {DLorem and {\v{C}}esk\'{a} republikd},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:e,
  title   = {ELorem and {{\v C}}esk\'a republikc},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:f,
  title   = {FLorem and {{\v{C}}}esk\'{a} republikd},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:a,
  title   = {AIpsum and {{\'I}sland}},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:b,
  title   = {BIpsum and {{\'{I}}sland}},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:c,
  title   = {CIpsum and {\'I}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:d,
  title   = {DIpsum and {\'{I}}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:e,
  title   = {EIpsum and {{\'I}}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:f,
  title   = {FIpsum and {{\'{I}}}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
\end{filecontents}

\begin{document}
\nocite{*}
\bibliographystyle{plain}
\bibliography{\jobname}
\end{document}

with (run with biblatex 3.15 dev for the expl3 case changer, the latex2e implementation has some quirks with non-ASCII chars in pdfLaTeX; alternatively use a Unicode engine)
\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{csquotes}
\usepackage[casechanger=expl3]{biblatex}

\DeclareFieldFormat{titlecase}{\MakeSentenceCase*{#1}}

\begin{filecontents}[force]{\jobname.bib}
@article{lorem:a,
  title   = {ALorem and {{\v C}esk\'a} republika},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:b,
  title   = {BLorem and {{\v{C}}esk\'{a}} republikb},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:c,
  title   = {CLorem and {\v C}esk\'a republikc},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:d,
  title   = {DLorem and {\v{C}}esk\'{a} republikd},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:e,
  title   = {ELorem and {{\v C}}esk\'a republikc},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{lorem:f,
  title   = {FLorem and {{\v{C}}}esk\'{a} republikd},
  author  = {Anne Uthor},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:a,
  title   = {AIpsum and {{\'I}sland}},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:b,
  title   = {BIpsum and {{\'{I}}sland}},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:c,
  title   = {CIpsum and {\'I}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:d,
  title   = {DIpsum and {\'{I}}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:e,
  title   = {EIpsum and {{\'I}}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
@article{ipsum:f,
  title   = {FIpsum and {{\'{I}}}sland},
  author  = {Anne N. Other},
  journal = {Journal},
  year    = {1995},
  volume  = {352},
}
\end{filecontents}
\addbibresource{\jobname.bib}

\begin{document}
\nocite{*}
\printbibliography
\end{document}

Note how the D cases are protected against case change with Biber, while they are not case protected with BibTeX. If Biber were to strip the additional braces here, we would get the same result as in BibTeX (compare the C cases).

I had an idea about this I've not tried before - now implemented in DEV.

All of 'my' cases and the ADS export (cf. #297) work brilliantly!
I don' know if it's related, but https://github.com/plk/biblatex/issues/727 https://github.com/plk/biber/issues/216 seems to be an issue again.
\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{csquotes}

\usepackage[style=authoryear, backend=biber]{biblatex}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@online{fontconfig,
  author ={{\texttt{freedesktop.org}}},
  sortname = {freedesktop},
  title = {Fontconfig},
  subtitle = {A library for configuring and customizing font access},
  date = {2016-06-15},
  urldate={2017-03-18},
  url = {https://www.freedesktop.org/wiki/Software/fontconfig/}
}
@online{wikipedia,
  author    = {{\WikipediA}},
  sortlabel = {Wikipedia},
  sortname  = {Wikipedia},
  title     = {Lucida},
  date      = {2016-10-19},
  urldate   = {2017-04-03},
  url       = {https://en.wikipedia.org/wiki/Lucida},
}

@online{features,
  author    = {{\WikipediA}},
  sortlabel = {Wikipedia},
  sortname  = {Wikipedia},
  title     = {List of typographic features},
  date      = {2017-02-21},
  urldate   = {2017-03-24},
  url       = {https://en.wikipedia.org/wiki/List_of_typographic_features},
}
\end{filecontents}

\def\WikipediA{wikipedia}

\addbibresource{\jobname.bib}

\begin{document}
\nocite{*}
\printbibliography
\end{document}

gives bad familyi's
      \name{author}{1}{}{%
        {{un=0,uniquepart=base,hash=f54ec09db02860f10fd50e2ce18d24db}{%
           family={{\texttt{freedesktop.org}}},
           familyi={f\bibinitperiod}}}%
      }
...
      \name{author}{1}{}{%
        {{un=0,uniquepart=base,hash=95387a9e1a6bf37286493c821a0b17da}{%
           family={{\WikipediA}},
           familyi={}\bibinitperiod}}}%
      }
...
      \name{author}{1}{}{%
        {{un=0,uniquepart=base,hash=95387a9e1a6bf37286493c821a0b17da}{%
           family={{\WikipediA}},
           familyi={}\bibinitperiod}}}%
      }

Please try 2.15 now - this was really something in the initials generation for edge cases.

The errors are gone!
I'm a bit concerned about the change from
\field{title}{Signs of W$\frac{o}{a}$nder}

to
\field{title}{Signs of W$\frac{o}a$nder}

but the versions are equivalent (even though the former is much nicer).

That is a bit ugly but only happens with single characters in braces, which should be equivalent. Multi-argument macros are a huge pain to deal with here but are rare enough.

I think one of the consequences of your recent changes is to remove brackets around all single characters. This is not a good idea, e.g.
@BOOK{xxx,
  AUTHOR = {X, Y},
  DATE = {2020},
  TITLE = {Part {I}},
}

will be turned into
@BOOK{xxx,
  AUTHOR = {X, Y},
  DATE = {2020},
  TITLE = {Part I},
}

by biber --quiet --tool --outfile test2.bib test.bib. And therefore "I" maybe "i" in some citations styles.

This is a constant problem I'm afraid - the latex decoding in biber is syntactical and it's impossible to cover all cases. Generally, things like part numbers should be in fields like volume or number. Another way around this specific example is to use double-braces: {{I}}. This edge-case only occurs with single glyphs in braces and doesn't affect, for example {II}.

I know this is a pain, but is there absolutely no chance this might work?
I think the rule here is that brace stripping should only happen if the braced contents start with a backslash, or more precisely with a macro encoding the macro version of a non-ASCII char.
I can write up some more systematic tests if you like.

I am not sure what your interpretation of "constant" is. The problem was not present in previous versions (let's say a few days ago?) and it breaks compatibility with bibtex. I have several books, journals and papers in my bibtex-file that have single capital letters in their respective titles, either due numbering or units e.g. "T" for Tesla. I guess removing brackets only in the case of following backslash should be save (since titlecase does not matter anyway here).

Ah, one other case in which we probably want braces removed is when an empty group follows a macro that Biber encoded into UTF-8 character (it appears possible to add code to capture the empty group to the code that searches these  macros and replaces them with UTF-8 chars)

The problem is that fixing one edge case breaks others fairly reliably in this area because we are doing this with (very complex) regexps as there is no other way short of having a full TeX parser in there. It looks simple to differentiate between \'{I} and {I} but since we have no semantics here, there is no way of knowing if a preceding macro takes any arguments or how many arguments it takes. This is always going to be a hack - so far it has been a reasonably good one but the edge cases mount and the new capitalisation code meant we had to shift the problem elsewhere. I don't like it either and will have to think about it more.

I'd have thought that it might be possible to get the few cases right that we need to get right. If we ignore initial generation, which is problematic with macros anyway,  for the moment, we don't need brace stripping for all macros, we only need it for diacritic-macros like \'. Biber already knows that \' does something to the next char and can already ignore the braces around it.

I'll have a look - it's just a little unsatisfactory having to break out into special cases when I've tried to keep this module as general as possible but I don't think there is much choice even though it makes things harder to maintain ...

Here are a few test cases I compiled. I will probably add to this if I find more, but I wanted to post it now in case I forgot about it.
\documentclass[british]{article}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{csquotes}

\usepackage[style=authoryear, backend=biber]{biblatex}

\renewbibmacro*{finentry}{%
  \setunit{\newline}\printfield{verba}%
  \setunit{\newline}\printfield{verbb}%
  \finentry}

\begin{filecontents}[force]{\jobname.bib}
@book{appleby,
  author  = {Humphrey Appleby},
  title   = {Harmless uses {I} {II} {Humphrey Appleby} {H}umphrey {A}ppleby},
  verba   = {Raw: {I} {II} {Humphrey Appleby} {H}umphrey {A}ppleby},
  verbb   = {Expected: {I} {II} {Humphrey Appleby} {H}umphrey {A}ppleby},
  date    = {1981},
}
@book{bppleby:b,
  author  = {Humphrey Bppleby},
  title   = {Single letter (BibTeX style)
             {\"a} {\"{o}} {\v C} {\v{Z}}},
  verba   = {Raw: {\"a} {\"{o}} {\v C} {\v{Z}}},
  verbb   = {Expected: ä ö Č Ž},
  date    = {1982},
}
@book{bppleby:l,
  author  = {Humphrey Bppleby},
  title   = {Single letter (LaTeX style)
             \"a \"{o} \v C \v{Z}},
  verba   = {Raw: \"a \"{o} \v C \v{Z}},
  verbb   = {Expected: ä ö Č Ž},
  date    = {1982},
}
@book{cppleby:b,
  author  = {Humphrey Cppleby},
  title   = {Protected single letter (BibTeX style)
             {{\"a}} {{\"{o}}} {{\v C}} {{\v{Z}}}},
  verba   = {Raw: {{\"a}} {{\"{o}}} {{\v C}} {{\v{Z}}}},
  verbb   = {Expected: {ä} {ö} {Č} {Ž}},
  date    = {1983},
}
@book{cppleby:l,
  author  = {Humphrey Cppleby},
  title   = {Protected single letter (LaTeX style)
             -- doesn't exist},
  date    = {1983},
}
@book{dppleby:b,
  author  = {Humphrey Dppleby},
  title   = {Words (BibTeX) {\"a}s{\"a}n {\"{o}}l{\"{o}}n
             {\v C}e{\v c}en {\v{Z}}e{\v{z}}en},
  verba   = {Raw: {\"a}s{\"a}n {\"{o}}l{\"{o}}n
             {\v C}e{\v c}en {\v{Z}}e{\v{z}}en},
  verbb   = {Expected: äsän ölön Čečen Žežen},
  date    = {1984},
}
@book{dppleby:l,
  author  = {Humphrey Dppleby},
  title   = {Words (LaTeX) \"as\"an \"{o}l\"{o}n \v Ce\v cen \v{Z}e\v{z}en},
  verba   = {Raw: \"as\"an \"{o}l\"{o}n \v Ce\v cen \v{Z}e\v{z}en},
  verbb   = {Expected: äsän ölön Čečen Žežen},
  date    = {1984},
}
@book{eppleby:b,
  author  = {Humphrey Eppleby},
  title   = {Protected Words (BibTeX) {{\"a}s{\"a}n} {{\"{o}}l{\"{o}}n}
             {{\v C}e{\v c}en} {{\v{Z}}e{\v{z}}en}},
  verba   = {Raw: {{\"a}s{\"a}n} {{\"{o}}l{\"{o}}n}
             {{\v C}e{\v c}en} {{\v{Z}}e{\v{z}}en}},
  verbb   = {Expected: {äsän} {ölön} {Čečen} {Žežen}},
  date    = {1985},
}
@book{eppleby:l,
  author  = {Humphrey Eppleby},
  title   = {Protected Words (LaTeX) {\"as\"an} {\"{o}l\"{o}n}
             {\v Ce\v cen} {\v{Z}e\v{z}en}},
  verba   = {Raw: {\"as\"an} {\"{o}l\"{o}n}
             {\v Ce\v cen} {\v{Z}e\v{z}en}},
  verbb   = {Expected: {äsän} {ölön} {Čečen} {Žežen}},
  date    = {1985},
}
@book{fppleby,
  author  = {Humphrey Fppleby},
  title   = {Macros \emph{Hullo} $\frac{a}{b}$},
  verba   = {Raw: \emph{Hullo} $\frac{a}{b}$},
  verbb   = {Expected: \emph{Hullo} $\frac{a}{b}$},
  date    = {1986},
}
@book{gppleby:b,
  author  = {Humphrey Gppleby},
  title   = {Macros (BibTeX) \emph{H{\"u}llo} \emph{H{\"{e}}llo}
             \emph{{\v C}e{\v c}en} \emph{{\v{Z}}e{\v{Z}}en}},
  verba   = {Raw:  \emph{H{\"u}llo} \emph{H{\"{e}}llo}
             \emph{{\v C}e{\v c}en} \emph{{\v{Z}}e{\v{Z}}en}},
  verbb   = {Expected: \emph{Hüllo} and \emph{Hëllo} \emph{Čečen} \emph{Žežen}},
  date    = {1987},
}
@book{gppleby:l,
  author  = {Humphrey Gppleby},
  title   = {Macros (LaTeX) \emph{H\"ullo} \emph{H\"{e}llo}
             \emph{\v Ce\v cen} \emph{\v{Z}e\v{Z}en}},
  verba   = {Raw:  \emph{H\"ullo} \emph{H\"{e}llo}
             \emph{\v Ce\v cen} \emph{\v{Z}e\v{Z}en}},
  verbb   = {Expected:  \emph{Hüllo} and \emph{Hëllo} \emph{Čečen} \emph{Žežen}},
  date    = {1987},
}
@book{hppleby:b,
  author  = {Humphrey Hppleby},
  title   = {Protected macros (BibTeX) {\emph{H{\"u}llo}}{\emph{H{\"{e}}llo}}
             {\emph{{\v C}e{\v c}en}} {\emph{{\v{Z}}e{\v{Z}}en}}},
  verba   = {Raw: {\emph{H{\"u}llo}} {\emph{H{\"{e}}llo}}
             {\emph{{\v C}e{\v c}en}} {\emph{{\v{Z}}e{\v{Z}}en}}},
  verbb   = {Expected:  {\emph{Hüllo}} and {\emph{Hëllo}} {\emph{Čečen}} {\emph{Žežen}}},
  date    = {1988},
}
@book{hppleby:l,
  author  = {Humphrey Hppleby},
  title   = {Protected macros (LaTeX) {\emph{H\"ullo}} {\emph{H\"{e}llo}}
             {\emph{\v Ce\v cen}} {\emph{\v{Z}e\v{Z}en}}},
  verba   = {Raw: {\emph{H\"ullo}} {\emph{H\"{e}llo}}
             {\emph{\v Ce\v cen}} {\emph{\v{Z}e\v{Z}en}}},
  verbb   = {Expected: {\emph{Hüllo}} and {\emph{Hëllo}}  {\emph{Čečen}} {\emph{Žežen}}},
  date    = {1988},
}
\end{filecontents}
\addbibresource{\jobname.bib}

\begin{document}
\nocite{*}
\raggedright
\printbibliography
\end{document}

Please try DEV now - I put a fix in which addresses single-char diacritic macros which fixes the Part {I} example.

Thanks a lot! I checked the new binary with my rather large biblatex collection (50k lines) and there is only one (minor) issue with {IEEE} {CG}\&{A} (short title of the IEEE Computer Graphics and Applications journal): it becomes {IEEE} {CG}\&A. Not sure if it makes sense to hunt this particular issue down...  personally, I can work around it, e.g. by adding spaces to the short title {IEEE} {CG} \& {A} - not great but ok!

Hmm, that's a strange edge case indeed - there really should be a space there but that wouldn't help with something legitimate like \&\;{A}. Please try 2.15 now - I think this should also be resolved now.

It is astonishing how many edge cases come out of the woodwork here. All my tests look great (but as becoming more and more obvious my tests don't even scratch the surface of what people have in their .bib files).
Just for the record A\&B is valid TeX and differs from A\& B in output. Space is only removed after control words consisting of (TeX) letters (characters of catcode 11) and the control space \ . Control sequences consisting of a single non-letter character do not skip the following space.
\documentclass{article}

\begin{document}
A\&B

A\& B

A\,B

A\, B
\end{document}

Please try 2.15 now - I think this should also be resolved now.

Yes, perfect! I confirm that the new version does not strip any intended bracket. By the way I am using 2.15dev for quite some time now and I am very happy with it.... Thanks!

@gerking Please see https://github.com/plk/biber/issues/329. But really I think these braces are excessive when they are added to all entries with hyphenated given names.

@gerking Please see #329. But really I think these braces are excessive when they are added to all entries with hyphenated given names.

Thanks, and sorry for duplicating.


[Feature] Location Suggestion for Weather App

Description:
The current weather app lacks a convenient way for users to input their location. Manually typing the location can be error-prone and time-consuming. This issue is raised to propose the addition of a location suggestion feature to enhance user experience.
Something like

I will find the suitable API for that and will also integrate the feature with both input fields (selecting city from dropdown would replace both the city and country input fields accordingly)

Please assign me this issue under iwoc2024

seems interesting go ahead assigning it to you

since i have to find the API and also integrate it well with the webapp. I think it should be hard
Making a footer also got the medium label, dual work shouldn’t be hard?
Let me know your views on it..
Thank you ;)

Making it hard raise a PR.


Unhandled KeyboardInterrupt exceptions on Unix when using the spawn start method for multiprocessing

What is wrong?
When using the spawn method (our default) for the multiprocessing module, I often get a bunch of unhandled KeyboardInterrupt exceptions when I Ctrl+C trinity. If using TRINITY_MP_CONTEXT=fork I never seem to get those
    INFO  06-04 15:06:13       chain  Imported 192 headers in 2.25 seconds, new head: #1346212 (1cd32f)
^C    INFO  06-04 15:06:14        main  Keyboard Interrupt: Stopping
Process Process-2:3:
Process Process-2:1:
    INFO  06-04 15:06:14         ipc  Sent SIGINT to process 27860, waiting 5 seconds for it to terminate
Process Process-2:2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/concurrent/futures/process.py", line 169, in _process_worker
    call_item = call_queue.get(block=True)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 93, in get
    with self._rlock:
  File "/usr/lib/python3.6/multiprocessing/synchronize.py", line 96, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/concurrent/futures/process.py", line 169, in _process_worker
    call_item = call_queue.get(block=True)
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 94, in get
    res = self._recv_bytes()
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt

it looks like the kind of thing that would be caused by the multiprocessing module:

If the SIGINT signal generated by Ctrl-C arrives while the main thread is blocked by a call to BoundedSemaphore.acquire(),
Lock.acquire(), RLock.acquire(), Semaphore.acquire(), Condition.acquire() or Condition.wait() then the call will be immediately
interrupted and KeyboardInterrupt will be raised.

In trying to remember why spawn was chosen, this is the best I could find.
https://docs.python.org/3.6/library/multiprocessing.html#contexts-and-start-methods
If we use fork it looks like we won't be usable on windows platforms.

Fixed in https://github.com/ethereum/py-evm/pull/1177


Ensure user secrets ID does not contain invalid file name characters

Resolves https://github.com/aspnet/DotNetTools/issues/229
cc @divega

Fix seems fine to me, though I think the value is minimal.

cc @HaoK does this test failure on Linux look familiar?
fail:     Microsoft.Extensions.Configuration.Test.ConfigurationTests.CreatingOptionalFileInNonExistentDirectoryWillReload [FAIL]
info:       Assert.Equal() Failure
info:       Expected: JsonValue1
info:       Actual:   (null)
info:       Stack Trace:
info:         /home/travis/build/aspnet/Configuration/test/Microsoft.Extensions.Configuration.FunctionalTests/ConfigurationTests.cs(612,0): at Microsoft.Extensions.Configuration.Test.ConfigurationTests.<CreatingOptionalFileInNonExistentDirectoryWillReload>d__33.MoveNext()

Yes its muted and flaky, its tracked by https://github.com/aspnet/Configuration/issues/468 and safe to ignore

Ok, thanks.

Ping. Ok to merge?


strange behavior  with GET param

Hello,
thank for the gem, it 's  helpfull.
I have problem with spec like:
  let(:account) { Account.last }
  let(:authorized_admin) { account.supervisors.first }
  let(:incident) { account.incidents.last }

  describe "GET /api/incidents/:id" do
    context "when user is NOT authorized" do
      it "Kick off unauthenticated/unauthorized user" do
        get "/api/incidents/#{incident.id}", headers: valid_headers.except("Authorization"), params: { version: 2 }

        expect(response.body).to eq "You can not request this API until valid Authorization token is provided"

        get "/api/incidents/#{incident.id}", headers: valid_headers.merge({ "Authorization" => "Bearer InvalidToken" })

        expect(response.body).to eq "You can not request this API until valid Authorization token is provided"
      end
    end

    context "when user is authorized" do
      context "when incident exists" do
        it "returns incident as json" do
          expected_response_body = incident_to_json(incident)

          get "/api/incidents/#{incident.id}", headers: valid_headers, params: { version: 2 }

          expect(response).to have_http_status(:ok)
          expect(response.headers["Content-Type"]).to eq("application/json; charset=utf-8")
          expect(response.body).to eq expected_response_body
        end
      end

      context "when incident do not exists at all" do
        it "returns 404" do
          incident_id = Incident.maximum(:id) + 1

          get "/api/incidents/#{incident_id}", headers: valid_headers, params: { version: 2 }

          expect(response).to have_http_status(:not_found)
        end
      end
    end
  end

When run with OPENAPI=1 it creates this  specification (shortened):
---
openapi: 3.0.3
info:
  title: some  app
  version: 2.0.0
paths:
  "/api/incidents/{id(}/{version)}":
    get:
      summary: show
      tags:
      - API::Versioning
      parameters:
      - name: id
        in: path
        required: true
        schema:
          type: integer
        example: 60
      - name: version
        in: query
        schema:
          type: integer
        example: 2
      responses:
        '200':
           ..... all attributes  and so on.....

See  "/api/incidents/{id(}/{version)}": path.
I am expecting  "/api/incidents/{id}"path and listedversionparameter inparameters` key (it is,correctly, there).
Do I do something wrong?

No activities for a while, so let me close this until this becomes actionable on my end. I'll reopen this once I have a way to reproduce this. Thank you.


Fix splitting text nodes when moving caret in contenteditable

I know this repository is deprecated, but some people may be using it.
@yuku-t, there is a fix for contenteditable. If, by any chance, you could merge it, I would really appreciate it.
Thank you

@mvlabat v1.8.3 is released

Thank you very much!


how to get x,y,tx,ty,allx,ally,graph

Hi, how should i get x,y,tx,ty,allx,ally,graph? thk

In main.ipynb, you can get features, adj, labels, and split_train_ids.


Please yank 1.4.1 on npmjs.com

Please yank 1.4.1 on npmjs.com, it confuses a lot of services like @dependabot. See also https://github.com/elm-lang/elm-platform/issues/237

@ream88 I've marked 1.4.1 as deprecated. Can you check if that fixed it, and if so, close this issue?
Thanks!

@rtfeldman thx man, however I’m out of the office for a week. Will check it then!

ok, I'm gonna close then... please reopen if it's still happening!

It seems to work now, closing this. Thx again!


How to have multiple gravity to the tooltip?

I am trying to place a tooltip to the bottom-right of a view, which happens to be at the right most side of the screen.
However, not using fitToScreen ends up with tooltip being outside of the screen, where an xOffset needs to be set, however this offset is not very reliable since the text can change in the tooltip.
Using fitToScreen also ends up creating a problem where tooltip automatically is set to Gravity.LEFT since bottom is not an option ( because it tries to draw it outside of the screen )
Here is a piece of code:
 val width = resources.displayMetrics.widthPixels * 2  / 3
        val tooltipPaddingRight = resources.getDimensionPixelSize(R.dimen.tooltip_margin_right)
        val tooltip = Tooltip.Builder(requireContext())
                .anchor(imageViewAction,-width / 2 + tooltipPaddingRight,0, false)
                .text("Tooltip message here")
                .maxWidth(width)
                .arrow(false)
                .styleId(R.style.SomeStyle)
                .showDuration(3000)
                .closePolicy(ClosePolicy.TOUCH_ANYWHERE_NO_CONSUME)
                .overlay(false)
                .create()

        tooltip.show(plusButton, Tooltip.Gravity.BOTTOM,true)

@tipialican
Hello, I have 2 ideas.  I hope this will help.

Set gravity as Tooltip.Gravity.RIGHT then add Y offset like .anchor(imageViewAction,0,48, false).
Make invisible view under plusButton then set it as anchor and set gravity as Tooltip.Gravity.RIGHT


Handle grayscale images and Theano vs. TF channel location difference

Two features have been added:

images can be grayscale. Previously the input channels was assumed to be 3, which causes models to fail that expect 1 input channel.
Tensorflow and Theano use different ordering for the channel dimension. This fix corrects for it.

sorry.. was working on removing it as I got this
On Fri, Nov 18, 2016 at 10:58 AM, Jake Bian<EMAIL_ADDRESS>wrote:

@jakebian commented on this pull request.
In npm-debug.log
https://github.com/jakebian/quiver/pull/25#pullrequestreview-9285024:

+3 info using<EMAIL_ADDRESS>+4 verbose stack Error: ENOENT: no such file or directory, open '/Users/gebhardt/git/quiver/package.json'
+4 verbose stack     at Error (native)
+5 verbose cwd /Users/gebhardt/git/quiver
+6 error Darwin 16.1.0
+7 error argv "/usr/local/Cellar/node/5.2.0/bin/node" "/usr/local/bin/npm" "start"
+8 error node v5.2.0
+9 error npm  v3.3.12
+10 error path /Users/gebhardt/git/quiver/package.json
+11 error code ENOENT
+12 error errno -2
+13 error syscall open
+14 error enoent ENOENT: no such file or directory, open '/Users/gebhardt/git/quiver/package.json'
+14 error enoent This is most likely not a problem with npm itself
+14 error enoent and is related to npm not being able to find a file.
+15 verbose exit [ -2, true ]

Remove the file please!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jakebian/quiver/pull/25#pullrequestreview-9285024,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD16XOlqhVtlPHgvorTfw1qlRqxb4H5Iks5q_fVtgaJpZM4K2tCc
.

Great, can you squash the last 3 commits though? The last 2 commits are not very meaningful.

Awesome - I'll test it out before merging. Thanks @dan-1d!


Pre-creation network access validation

I've heard about some folk having a bad experience spinning up ADOP from a location with various ports blocked.  Has anyone experienced this and if so what failed and when?  Obviously if we can move the failures earlier in provisioning people will be happier.

@kramos I also find ports to be the biggest problem.

Docker daemon port (TCP 2376) is often blocked - fails at docker-compose stage.
SSH port blocked - fails at docker installation stage after docker-machine has created an instance.

This is unrelated to the ADOP provisioning process. Features of the release note are also sometimes blocked (see Accenture#nginx-28 ).

How about something like this:
curl portquiz.net:2376 || echo ERROR: cannot connect outbound on port 2376 so won’t be able to access Docker. && exit 1
Only snag is if portquiz.net is down or blocked you could get a false negative.

@RobertNorthard @kramos
Bash has the capability to access TCP and UDP ports...
as per my raspberry pi:
pi@raspberrypi:~ $ cat < /dev/tcp/<IP_ADDRESS>/22 SSH-2.0-OpenSSH_6.7p1 Raspbian-5+deb8u3 ^C
So we can potentially use this to see if we get a response from said ports right?

I'm facing the same issue, both when I execute from my mac and when I execute from an EC2 instance.
Mac:
Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "<IP_ADDRESS>:2376": dial tcp <IP_ADDRESS>:2376: getsockopt: connection refused
EC2, has a slightly different error:
Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "<IP_ADDRESS>:2376": read tcp <IP_ADDRESS>:39556-><IP_ADDRESS>:2376: read: connection reset by peer

Both machines have said port open... thoughts?

@chad-mott can you reach port 2376 on the target instance from the host you're running the command from?

Yes
On Dec 20, 2016, at 4:19 PM, Luis Sousa <notifications<EMAIL_ADDRESS>wrote:
@chad-motthttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_chad-2Dmott&d=DgMCaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=awfRDlSyeo0JIr3RHBE09mxTLabSR6JInqgcT7beGlM&m=jC_q5WTgGZ4AzHUVqMKZFLNHh9hAVdJhbCRDoTaoRlY&s=7rxA3l9BBCV1O0S58iRmFY2hQiuyt2-d9fusdWoLu1g&e= can you reach port 2376 on the target instance from the host you're running the command from?

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Accenture_adop-2Ddocker-2Dcompose_issues_172-23issuecomment-2D268375803&d=DgMCaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=awfRDlSyeo0JIr3RHBE09mxTLabSR6JInqgcT7beGlM&m=jC_q5WTgGZ4AzHUVqMKZFLNHh9hAVdJhbCRDoTaoRlY&s=7djKgauU8BP-u0_OXWEH_ZWXekfmhBchJM_BhoU3fXE&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWaxhdHkLJ1HM7D8crXq-5FuNZH9yhi-5Ffzks5rKFRigaJpZM4LI8iR&d=DgMCaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=awfRDlSyeo0JIr3RHBE09mxTLabSR6JInqgcT7beGlM&m=jC_q5WTgGZ4AzHUVqMKZFLNHh9hAVdJhbCRDoTaoRlY&s=yWmLqoFh-3MqKULGGPZiCerZEeyWwMKZsvwfqwp3eno&e=.

This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.

www.accenture.com

also, not in the readme, but ./adop compose -i localhost init from a running instance works

FYI, this works now using docker machine v0.9.0-rc2
To validate, I downgraded back to 0.8.2, and it failed again.


#434 Changed If-Modified-Since behavior

If-Modified-Since returns the requested data if there's new data since. "started" param now controls which data to return.

LGTM. 🚢


Slim 762 il push alarm not

SonarQube analysis reported 1 issue

 1 critical

Watch the comments in this conversation to review them.

All is well.

All is well.

SonarQube analysis reported 7 issues

 5 major
 2 info

Watch the comments in this conversation to review them.
3 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 SmartMeteringManagementEndpoint.java#L336: Remove this unused method parameter "messagePriority".
 SmartMeteringManagementEndpoint.java#L320: Complete the task associated to this TODO comment.
 SmartMeteringManagementEndpoint.java#L366: Complete the task associated to this TODO comment.

All is well.

SonarQube analysis reported 7 issues

 5 major
 2 info

Watch the comments in this conversation to review them.
3 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 SmartMeteringManagementEndpoint.java#L336: Remove this unused method parameter "messagePriority".
 SmartMeteringManagementEndpoint.java#L320: Complete the task associated to this TODO comment.
 SmartMeteringManagementEndpoint.java#L366: Complete the task associated to this TODO comment.

All is well.

SonarQube analysis reported 3 issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 SmartMeteringManagementEndpoint.java#L336: Remove this unused method parameter "messagePriority".
 SmartMeteringManagementEndpoint.java#L320: Complete the task associated to this TODO comment.
 SmartMeteringManagementEndpoint.java#L366: Complete the task associated to this TODO comment.

All is well.

This pull request makes the build fail.

SonarQube analysis reported 3 issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 SmartMeteringManagementEndpoint.java#L336: Remove this unused method parameter "messagePriority".
 SmartMeteringManagementEndpoint.java#L320: Complete the task associated to this TODO comment.
 SmartMeteringManagementEndpoint.java#L366: Complete the task associated to this TODO comment.

All is well.


API: onDimensionsChanged event

Tested this in Oni2's branch (needed it for editor resizing) - seems like it works 👍


Nice format for JSON-API

Hey all,
Thanks for your great work on this. I'm interested in documenting a JSON-API implementation using the jsoinapi-resources gem, and I'm wondering about the fact that attributes and relationships include a lot of repeated boilerplate, e.g. if I wanted to document some attributes on a user object:
with_options scope: [:data, :attributes] do
          parameter :email
end

I end up with a lot of repetition in my parameters, e.g. ["data", "attributes"][email], ["data", "attributes"][first_name], ["data", "attributes"][last_name], etc. rather than just listing e.g. ("email", "first_name", "last_name") and having it be understood that they were all stored in data[attributes] following the JSON-API schema. Is there a way to simplify the presentation?
Thanks,

I want to add? that now it is not easy to declare information about relationship. For example, to create a Comment I need to specify its Post and User:
POST /comments

{
  "data": {
    "attributes": {
      text: "Hello"
    },
    "relationships": {
      "post": { "id": 1, "type": "posts" },
      "user": { "id": 2, "type": "users" }
    }
  }
}

And in my request I have:

post "comments" do
  parameter :text, "String", required: true, scope: [:data, :attributes]
  parameter :id, "Integer", required: true, scope: [:data, :relationships, :post]
  parameter :id, "Integer", required: true, scope: [:data, :relationships, :user]

  let(:id) { post.id } # goes in both id parameters

  ...

Maybe it will be convenient to specify a method option of parameter, because now I can't use let mapping feature

@sineed I think a method addition would be great, that would solve a good deal of recurring issues. If you have the time, I would gladly look at a PR. I am not sure when I'll be able to get to this myself.

@oestrich yes, I think I can find some time to implement this

I found here that I can use the following declaration to accomplish my case:
let(:data_relationships_user_data_id) { user.id }
let(:data_relationships_post_data_id) { post.id }

Looks quite heavy but I think that this is problem of JSON API =)
Also from docs:

The value of scoped parameters can be set with both scoped (let(:order_item_item_id)) and unscoped (let(:item_id)) methods. It always searches for the scoped method first and falls back to the unscoped method.

For me it is more than enough

@oestrich I can make a PR anyway if you wish

I think having the ability to specify a method is still valuable to have. It would solve issues where someone has client as a parameter, RAD is not happy in those cases. I would still love to see a PR for this.

FYI, in https://github.com/zipmark/rspec_api_documentation/pull/313 (adds APIB support), I had to have a method somehow because APIB specs group docs by that. So I made it work but only for .apib documents though. Adding general support shouldn't be hard.


Add customizable toolbar and app header

I've received a request to allow customizing the "toolbar" (re-arranging and removing elements from it) as well as adding the ability to add an application header above the toolbar, similar to how the UI prior to v3 was. The application header would also have some amount of customization allowed as well.
This is primarily for organizations embedding SQLPad into other applications or portals.
Is this something others would find useful?
My initial thought on the approach would be to take the existing toolbar and break it out into smaller components, and initialize it with some config string. For example the following string would have the queries button on the left, schema button and connection selector centered, and menu button on the right:
SQLPAD_TOOLBAR="[queries_button] [spacer] [schema_button] [connection_selector] [spacer] [menu_button]"

For the application header this would be a bit more difficult, as you'd presumably want to customize the content (change title for example) and it'd be less interactive. This ends up looking kinda like JSX, but JSX isn't something that can easily be integrated after initial build of the SQLPad UI.
# this looks like JSX but not, and its not something I'd like to parse and implement
SQLPAD_APP_HEADER="[spacer] [app_title title='My company's SQLPad'] [spacer] [user_name]"
# maybe it should be a mustache/handlebars template instead?
SQLPAD_APP_HEADER="

My company's SQLPad

{{user.name}}
"

Alternatively, instead of having this something injected by config, efforts could be made to make it easier to provide these customizations in a fork and minimize conflict?

Hi @rickbergfalk, I would like to see this as a feature.

Here's a simple suggestion (I think). Add two configuration items:

SQLPAD_HTML_PROLOGUE
Any HTML code provided here will be injected at the top of the SQL page (inside div id="root"). It can be used for branding, A simple example:

Your content goes here

SQLPAD_HIDE_HEADER
If set to true, the header will not be displayed. SQLPad branding will be provided elsewhere.

So setting these two example together, you'll get something like this:

@eladeyal-intel is this a feature you're interested in as well?
I did something like that first part in a PR but held off merging. https://github.com/rickbergfalk/sqlpad/pull/549
That PR did end up going into a fork of SQLPad I'm maintaining, but the custom header work is eventually going to be replaced with a custom front-end. Setting the HTML via config, then sending it via server turned out to be too limited.
Alternatives were brought up such as following some sort of plugin model and have that put in the UI project to allow for more flexibility.
As things stand I'm not sure how widely used this would be, and what common use cases would be for it.

@activiz are you interested in the reordering of the buttons and inputs displayed in the toolbar/header or are you looking to add custom content like a logo? or both?

If there was an easy way to customise without needs of additional seveI would have used it.

Going to close this one out. This work was originally scoped for a custom SQLPad fork I was working on, and the customization didn't really pan out. Technically it worked, but it wasn't flexible enough.
Not sure to what customizations would be of interest to others, but I'm concerned it is something that could gradually snowball into something difficult to maintain or lots of different config... and I find myself with less and less time these days


openmp package not found on OSX

Solution to issue cannot be found in the documentation.

[X] I checked the documentation.

Issue
I've included openmp as a dependency in my CI for many years and a recent build started failing with:
Updating 'opty-dev' env from conda env update...
  /Users/runner/miniconda3/condabin/mamba env update --name opty-dev --file /Users/runner/work/opty/opty/setup-miniconda-patched-opty-dev-env.yml
  Channels:
   - conda-forge
   - defaults
  Platform: osx-arm64
  Collecting package metadata (repodata.json): ...working... done
  Solving environment: ...working... failed
  Warning:
  PackagesNotFoundError: The following packages are not available from current channels:

    - openmp

  Current channels:

    - https://conda.anaconda.org/conda-forge/osx-arm64
    - https://repo.anaconda.com/pkgs/main/osx-arm64
    - https://repo.anaconda.com/pkgs/r/osx-arm64

  To search for alternate channels that may provide the conda package you're
  looking for, navigate to

      https://anaconda.org/

  and use the search bar at the top of the page.

This comes from the builds here: https://github.com/csu-hmc/opty/pull/158
And the env.yml file is:
name: opty-dev
channels:
  - conda-forge
dependencies:
  - coverage
  - cyipopt >=1.1.0
  - cython >=0.29.19
  - matplotlib >=3.2.0
  - numpy >=1.19.0
  - numpydoc
  - openmp
  - pytest
  - pytest-cov
  - scipy >=1.5.0
  - sphinx
  - sphinx-gallery
  - sympy >=1.6.0

I'm not sure why openmp can't be found for osx-arm64, as it was able to just a week ago in the same CI runs. Would this error be shown for a package combination clash? Or only if the package cannot be found at all on anaconda.org?
Installed packages
name: opty-dev
channels:
  - conda-forge
dependencies:
  - coverage
  - cyipopt >=1.1.0
  - cython >=0.29.19
  - matplotlib >=3.2.0
  - numpy >=1.19.0
  - numpydoc
  - openmp
  - pytest
  - pytest-cov
  - scipy >=1.5.0
  - sphinx
  - sphinx-gallery
  - sympy >=1.6.0

### Environment info

```shell
NA, running on CI with setup-minconda action.

I'm thinking this may be a package clash. setup-miniconda also adds python=3.9 to the env.yml file, so maybe that is doing something fishy.

I determined this occurred when switching from setup-minconda@v2 to v3. Closing, as it isn't likely an issue with this feedstock.

The package is named llvm-openmp, you need to use that name.

Thanks, I checked the history of this repo and it seems the name changed from openmp to llvm-openmp at some point. When I use the name llvm-openmp in my build conda can't find it for windows (whereas the original issue was not finding openmp for osxarm64).

There's no question that llvm-openmp is built & published for all the platforms conda-forge supports. If something is not working please ensure you have only conda-forge packages and use strict channel priority. If it's still a problem then, open a new issue with all the required information

The information is all in the issue I linked, in the notes and commits there. As I said, I closed this issue because the problem I had is most likely tied to different behavior of setup-miniconda action v2 and v3. I have not yet tried to build further with v3 and once I do, I will communicate any errors.

That's not how this works. I'm not going to go hunt in random links for the required information; if you need help (from unpaid volunteers), please file a proper bug report. At the very least that means filling out the required information in the issue template, and ideally doing some basic research.

I closed this issue because I did not plan to provide more information here. I pointed to the other issue because you seemed to want to know more.


Creating new quest from other plugin

Hi!
Is there a way to create quest in way other than editing then manually in yaml file and from gui? In my case I looked for method that I can call from my own plugin to generate quest.
Where else have you looked for answers to this question?
I've looked through code of plugin but it is hard to understand whole logic of a complex project. So if there is a simple solution, it'll be great to know about it before writing such method by myself.
Also, I read whole wiki and found out nothing about my problem.
Share your question with as much detail as possible?

Thanks for your question. Quests was not designed to allow other plugins to create quests programatically. The most you can do currently is create a new Quest object and set an ID, name, and other attributes. However, you will be unable to apply any Stages and it may not even be possible to save the quest without reflection.
As you may know, the Custom Quest API can be used to create Requirements, Rewards, and Objectives. This allows you to give server owners the ability to use your novel ideas as they see fit in their quests - and cease if/when they outgrow them. Circumventing this would possibly strip owners of this power, and questadmins of their usefulness. It is for this reason, as well a desire to prevent potential abuse, that the creation of quests is left in human hands.


Create new resources content has formatting issues

Describe the bug
When creating a new publishing profile, the "Create new resources" content has formatting issues when describing what's required. specifically, the azure subscription requirement.
Version
Version:  1.4.0-nightly.232669.4daad5c
Electron: 8.2.4
Chrome: 80.0.3987.165
NodeJS: 12.13.0
V8: 8.0.426.27-electron.0
Browser

[x] Electron distribution
[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

OS

[x] macOS
[ ] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:

create a new, or open an existing bot with Composer
Nav to the publish your bots page
Click the Add new link to invoke the Add a publishing profile flow
Name your profile, select the Azure publishing target
Log into azure if prompted
From the resource selection modal, select the "Create new resources"
Review the "Create New Resources" text.  Notice the formatting/layout issue with the second sentence.  see screenshot below.

Expected behavior
Proper formatting of the second sentence
Screenshots

I believe duplicate of #6825


Debian update

[x ] I'm reporting a broken site support
[ ] I've verified that I'm running youtube-dl version 2<IP_ADDRESS>
[ ] I've checked that all provided URLs are alive and playable in a browser
[ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[ ] I've searched the bugtracker for similar issues including closed ones

I've tried updating via sudo youtube-dl -U
But I get:
youtube-dl: error: youtube-dl's self-update mechanism is disabled on Debian
It seems as if the Debian repository still has youtube-dl (2019.01.17-1.1)
Does anyone know how I can enable the self-update mechanism?
Verbose log

PASTE VERBOSE LOG HERE

Description

WRITE DESCRIPTION HERE

By removing Debian version and installing official version.

Great thanks!


host.docker.internal is not working

Hi,
I am following the example of debugging with pycharm in sagemaker local mode, but unfortunately failed.
The error is "(socket.gaierror: [Errno -2] Name or service not known)". Docker version is  20.10.17 and OS is ubuntu 18.04.
Some addition information: For Mac, I have to change "host.docker.internal" to "docker.for.mac.host.internal" to make is work.
Any idea how to make it work on Linux?
Thanks in advance.

For those who are interested, I simply replace host.docker.internal by the Gateway address of the sagemaker-local network (via docker network inspect ${sagemaker-local} --format='{{( index .IPAM.Config 0).Gateway}}), and now it works as expected.


Check how prow dco plugin looks

What this PR does / why we need it:
This is just a test for the Prow DCO plugin. I added an unsigned commit to see how it looks.
/hold
/cc @rmohr
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:

NONE

Looking good so far, the only thing I need to know is how to make the check mandatory.

Maybe this is the way to do it: kubevirt/project-infra#192

Looking good so far, the only thing I need to know is how to make the check mandatory.

I think that for tide it is mandatory anyway, but for normal merges, I don't think so.

Ah, ok, then we don't need to make it mandatory like in the PR I just posted, right? Need to brush up reading about Prow config then :)

But shouldn't they also be marked as required then? I manually did that in branch protection settings to make it look like it is now.

But shouldn't they also be marked as required then? I manually did that in branch protection settings to make it look like it is now.

Afaik, tide only looks on its internal configuration. I think it even tries to merge PRs if a required job fails and it is not listed in the branch protection config.

Afaik, tide only looks on its internal configuration. I think it even tries to merge PRs if a required job fails and it is not listed in the branch protection config.

But: there exists the branchprotector https://github.com/kubernetes/test-infra/tree/master/prow/cmd/branchprotector (which we did not deploy), which can configure github the way prow sees the configuration.

But shouldn't they also be marked as required then? I manually did that in branch protection settings to make it look like it is now.

Afaik, tide only looks on its internal configuration. I think it even tries to merge PRs if a required job fails and it is not listed in the branch protection config.

I found this configuration of tide, might be the right one?

I found this configuration of tide, might be the right one?

Great, that looks like it!

/close
Everything works like expected. @dhiller thx!


Add pre-commit hook for secrets scanning

Issue #, if available:
N/A.
Description of changes:

A pre-commit hook has been added that uses aws-secrets in order to prevent secrets from being committed.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

nit:  Shall we provdie instructions that git-secrets should be pre-installed on the local machine?

nit: Shall we provdie instructions that git-secrets should be pre-installed on the local machine?

I have added a DEVELOPER_GUIDE. Users need to have pre-commit installed on their system, git-secrets will be retrieved by pre-commit.


Linguistic Analysis API

This Cognitive Service does not exist (anymore), please update the documentation accordingly

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f8a51dd9-65b7-5039-06d0-d096ad51d0bb
Version Independent ID: 9c2c99ed-8d17-7865-e680-2107b7f0ebf1
Content: Choosing a natural language processing technology
Content Source: docs/data-guide/technology-choices/natural-language-processing.md
Service: architecture-center
Sub-service: cloud-fundamentals
GitHub Login: @ZoinerTejada
Microsoft Alias: pnp

Hi @krusejan Thanks for your feedback. We will review and update as appropriate.

@krusejan We still have Azure Cognitive Services. The catalog of services within Azure Cognitive Services can be categorized into five main pillars - Vision, Speech, Language, Web Search, and Decision.
More detail can be found here https://docs.microsoft.com/en-us/azure/cognitive-services/welcome
Please let me know if I misunderstood the question. So for now I will proceed with closure of this and If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Typo

In Hand Tracking Gestures and Interactable buttons, step 4: the text says to set the Y position of the cube at 0.4, but screenshot shows 0.04 which is the right position to match the example.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5ff554d8-c68d-28a2-1331-3177bc8f6d71
Version Independent ID: a4419d7e-9355-76fd-4ca7-9ea26cb0a5aa
Content: Getting started tutorials - 3. Creating user interface and configure Mixed Reality Toolkit - Mixed Reality
Content Source: mixed-reality-docs/mrlearning-base-ch2.md
Product: mixed-reality
Technology: mr-dev
GitHub Login: @jessemcculloch
Microsoft Alias: jemccull

Resolved in #1953


Something is not working properly

Plugin version: <IP_ADDRESS>.1
Server version: 1.19.3
Server software: Paper
Client version (game version): 1.19.3
i cant disable The Scoreboard and the scoreboard is inside my hub
and its like flickering?
and i use DeluxeHub

Have you checked out the scoreboard.yml file? You can configure the scoreboards there, and if you want to disable any, just set it to an empty object. For example, if you want to disable the lobby scoreboard, just set it like this:
lobby: {}


adds tree-finder-breadcrumbs custom element, adds optional breadcrumbs to top of tree-finder elements

closes #9
Todo:

[x] add support for breadcrumbs to ContentsModel
[x] add factory function for breadcrumbs html contents
[x] add tree-finder-breadcrumbs custom element and TreeFinderBreadcrumbsElement implementation
[ ] add option to add tree-finder-breadcrumbs to top of tree-finder panel

still more work to do, but the basics are in place so merging and iterating


Error: Coudn't complete data download

Hi! Today I updated the arweave-js version on my project and notice an issue in my dApp.
I have this TX: A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0
And I'm using getData to get the data from the tx:
const data = await arweave.transactions.getData('A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0', {decode: true, string: true} );
Before the update everything was fine, the data was fetched, but now I'm getting these errors from the console:
[chunk] Failed to fetch chunk at offset<PHONE_NUMBER>2909
chunks.js:67 [chunk] This could indicate that the chunk wasn't uploaded or hasn't yet seeded properly to a particular gatway/node
Error: Error: Coudn't complete data download at 0/1563
I think that maybe the problem comes from the offset calculation and the arweave-js new update.
Please help! haha. Thanks in advance :)

If you take a look at
https://arweave.net/tx/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0/offset

you can track down where the chunks for a give transaction is located at.
The offset ends at<PHONE_NUMBER>4471 but begins at<PHONE_NUMBER>4471 - dataSize + 1.
So it starts at:<PHONE_NUMBER>4471 - 1563 + 1 =<PHONE_NUMBER>2909
Now if you try to download that chunk
curl https://arweave.net/chunk/11882184452909
<!DOCTYPE html>

<meta charset="utf-8">
<title>Error</title>

Not Found

it seems it isn't found.
this transaction was mined into block 756664 at 26th of August 2021 (writing this as I'm thinking out loud).
And indeed it is visible from the gateway cache: https://an7bym2iyfh2bmjinrfqykwm5sdjrqwos6kdgn7buhj6ku2m6twq.arweave.net/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0/
This means in simple terms, sadly, that the gateway received the chunks but didn't succeed in sending them to our nodes (that is a serious problem if true).
To verify this hypothesis: try to curl that chunk from peers in the network
https://arweave.net/peers
e.g.
curl http://<IP_ADDRESS>:1984/chunk/11882184452909
curl http://<IP_ADDRESS>:1984/chunk/11882184452909
curl http://<IP_ADDRESS>:1984/chunk/11882184452909
curl http://<IP_ADDRESS>:1984/chunk/11882184452909
To fix this:

download the data and the transaction headers.
Run the following code snippet as is described in README.md

const Arweave = require("arweave");
const ArweaveTransaction = require("arweave/lib/transaction.js");
const fs = require("fs");

const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

let data = fs.readFileSync(
  "./data_from_the_gateway.ext"
);

let txHeaders = require("./tx_header.json");

(async () => {
  const tx = new ArweaveTransaction.default(resumeObject);

  let uploader = await arweave.transactions.getUploader(tx, data);

  while (!uploader.isComplete) {
    await uploader.uploadChunk();
  }

})();

feel free to replace
const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

with a node or gateway of your choosing (given that arweave.net likely failed here, but I'm sure it will work the second time).
Alternative to all of this, you can choose to fetch the data from the gateway-cache instead of relying on getData:
const data = fetch("https://arweave.net/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0");

since the gateway cache isn't permanent, I strongly encourage to re-upload the chunks to make fully sure the nodes have them!
We are planing to re-upload all the chunks in coming weeks, but until then, we can only be as good as our weakest link, and it seems we clearly made a mistake here.

Thank you! I can understand the problem now. It seems that the chunk is not present. I will try the proposed solution, thanks in advance for your fast reply!

If you take a look at
https://arweave.net/tx/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0/offset

you can track down where the chunks for a give transaction is located at.
The offset ends at<PHONE_NUMBER>4471 but begins at<PHONE_NUMBER>4471 - dataSize + 1. So it starts at:<PHONE_NUMBER>4471 - 1563 + 1 =<PHONE_NUMBER>2909
Now if you try to download that chunk
curl https://arweave.net/chunk/11882184452909
<!DOCTYPE html>

<meta charset="utf-8">
<title>Error</title>

Not Found

it seems it isn't found.
this transaction was mined into block 756664 at 26th of August 2021 (writing this as I'm thinking out loud).
And indeed it is visible from the gateway cache: https://an7bym2iyfh2bmjinrfqykwm5sdjrqwos6kdgn7buhj6ku2m6twq.arweave.net/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0/
This means in simple terms, sadly, that the gateway received the chunks but didn't succeed in sending them to our nodes (that is a serious problem if true).
To verify this hypothesis: try to curl that chunk from peers in the network https://arweave.net/peers
e.g. curl http://<IP_ADDRESS>:1984/chunk/11882184452909 curl http://<IP_ADDRESS>:1984/chunk/11882184452909 curl http://<IP_ADDRESS>:1984/chunk/11882184452909 curl http://<IP_ADDRESS>:1984/chunk/11882184452909
To fix this:
1. download the data and the transaction headers.

2. Run the following code snippet as is described in README.md

const Arweave = require("arweave");
const ArweaveTransaction = require("arweave/lib/transaction.js");
const fs = require("fs");

const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

let data = fs.readFileSync(
  "./data_from_the_gateway.ext"
);

let txHeaders = require("./tx_header.json");

(async () => {
  const tx = new ArweaveTransaction.default(resumeObject);

  let uploader = await arweave.transactions.getUploader(tx, data);

  while (!uploader.isComplete) {
    await uploader.uploadChunk();
  }

})();

feel free to replace
const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

with a node or gateway of your choosing (given that arweave.net likely failed here, but I'm sure it will work the second time).
Alternative to all of this, you can choose to fetch the data from the gateway-cache instead of relying on getData:
const data = fetch("https://arweave.net/A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0");

since the gateway cache isn't permanent, I strongly encourage to re-upload the chunks to make fully sure the nodes have them!
We are planing to re-upload all the chunks in coming weeks, but until then, we can only be as good as our weakest link, and it seems we clearly made a mistake here.

I wonder if it would be worth putting this into advanced-readme.md or something maybe?

Hello team, probably was only a one in a million case. My dApp is working great again haha thank you very much for your fast support. I did what @hlolli suggested me and now everything seems to work fine.  This was my final code:

const Arweave = require("arweave");
const ArweaveTransaction = require("arweave/node/lib/transaction.js");
const fs = require("fs");

const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
});

let data = fs.readFileSync(
  "./txData.txt"
);

let txHeaders = require("./txHeaders.json");

(async () => {
  const tx = new ArweaveTransaction.default(txHeaders);

  let uploader = await arweave.transactions.getUploader(tx, data);

  while (!uploader.isComplete) {
    await uploader.uploadChunk();
  }

})();

Thanks again!

I did some digging around and came up with this, you can just feed it your txid 😃
		const txid = 'A34cM0jBT6CxKGxLDCrM7IaYws6XlDM34aHT5VNM9O0'

		const { data } = await axios.get('https://arweave.net/' + txid, {
			responseType: 'arraybuffer',
		})

		const uploader =  await arweave.transactions.getUploader(txid, data)

		while(!uploader.isComplete){
			await uploader.uploadChunk()
			console.log(`chunkUploaded: ${uploader.uploadedChunks}/${uploader.totalChunks} %${uploader.pctComplete}`)
			console.log(`lastResponseStats: ${uploader.lastResponseStatus}`)
		}

Big thank you @rosmcmahon Arweave still has so many mysteries to me. Community here is awesome, thank you both! You saved a good man today 👍🏽

I'm still encountering this issue while using SmartWeave. Weird experience.
While using the following node code block, I try to read the state of the following contract: e3f94a5pQrkoZHopWopR8vNFWDdrdt1uafxU1z8n6Yc
Note: the contract has been deployed using SmartWeave CLI
Node.js:
import { readContract } from "smartweave";
import Arweave from "arweave";

const CONTRACT = "e3f94a5pQrkoZHopWopR8vNFWDdrdt1uafxU1z8n6Yc";
const arweave = Arweave.init({
  host: "arweave.net",
  port: 443,
  protocol: "https",
  timeout: 20000,
  logging: false,
});

export async function readRegistryContract() {
  try {
    const state = await readContract(arweave, CONTRACT);
    return state
  } catch(error) {
    console.log(error);
    process.exit(1);
  }
}

/** PACKAGE.JSON Dependencies
  * "dependencies": {
   * "arweave": "^1.10.23",
   * "redstone-smartweave": "^0.4.4",
    *"smartweave": "^0.4.46"
 * }

 **/

However, while running the code above, the following error is thrown:

BUT
Using Smartweave-CLI to read the contract's state is a successful attempt:

The error was experienced also yesterday for this contract _PB9p2qoS8XbnBXL5UwuH2AYTSWVD6tI5rYYvq5G_MA but it's fine now.

i couldn't reproduce this using arweave.net. are you using some nodes directly?

Hi, the same issue happened to me when I  was testing.
txid: oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E (https://viewblock.io/arweave/tx/oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E)
I confirm the file uploaded by visiting:
https://arweave.net/oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E
but the getData API returns the same error shown above in this thread.
https://arweave.net/tx/oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E/offset
gives me:
{"size":"82","offset":"47767984578888"}
which I should be able to access via,
https://arweave.net/chunks/47767984578807
but returns an 400 error. any clues?

@yasushisakai

I confirm the file uploaded by visiting: https://arweave.net/oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E

unfortunately this only confirms that the data is cached in the arweave.net gateway.
your data is not seeded to the actual arweave network nodes for whatever reason. that's what getData is telling you above.
I have taken the liberty of running the reseeding code snippet solution above against your tx. and recieved these messages:
checking oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E ...
[chunk] Failed to fetch chunk at offset<PHONE_NUMBER>8807
[chunk] This could indicate that the chunk wasn't uploaded or hasn't yet seeded properly to a particular gatway/node
Error : Coudn't complete data download at 0/82 : Error: Coudn't complete data download at 0/82
    at Chunks.downloadChunkedData (/Users/ros/repo-personal/test-nodejs-harness/src/common/chunks.ts:77:15)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Transactions.getData (/Users/ros/repo-personal/test-nodejs-harness/src/common/transactions.ts:155:42)
    at async checkChunks (/Users/ros/repo-personal/test-nodejs-harness/src/ar-tx/reseed.ts:42:19)
    at async main (/Users/ros/repo-personal/test-nodejs-harness/src/ar-tx/reseed.ts:106:5)
reseeding using arweave.net cache oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E ...
82 bytes found in arweave.net cache
chunkUploaded: 1/1 %100
lastResponseStats: 200
you might need to wait 5-20 minutes for your reseeded data to become available.

so as explained above in this issue, your data was not seeded in this case.
as it was found in the arweave.net cache, this is probably due to an error at the time of posting to that gateway.
i hope this helps

update & fyi:
checking oYpB0tUDvYjphc_HxuFzzrrO4VVHozp7nmfFzCLZZ0E ...
82 bytes stored in the network nodes


[timeseries] Fix platform tests failing for Python 3.8 because of old numpy version

Description of changes:

Fix platform tests failing for Python 3.8 because numpy v1.24 gets installed on Python 3.8 envs

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/platform_tests ref=shchur:np-platform-test

Previous CI Run
Current CI Run


perf(lexer): low hanging fruits

From the v8 scanner blog, I still see low hanging fruits, let's play with these and make our lexer even faster!

Nerd sniping @YoniFeng, since you've already looked at the lexer a bit 😉

Performance issues are bugs for oxc. 👀

Performance is feature.

What low hanging fruit do you see? We can make a list of things to look at / try.
I read through it but didn't follow everything (for example the whitespace part where they saved a branch), so would need to take a better look.
But quick notes and thoughts:

Streaming vs reading whole file into memory: for Chromium it makes complete sense, it enables parallelization of parsing while data is still coming over the network ASAP.
For reading from disk - my guess is that it's negligible at best, but probably worse?
Might be worth trying for fun/learning, but I'd give it super low priority.
The code for the whitespace/comments SIMD would need to be altered so that it doesn't need all the bytes up front, anything else like that?

I didn't get the Whitepspace section just from reading. Will have to dig in to the code. I got that they managed to avoid a loop branch, but didn't fully comprehend the before/after. Also don't know if it's currently relevant for the lexer the way it currently is (given it currently skips all whitespace and doesn't return a whitespace token).
One assumption difference worth noting - v8 expects/optimizes for minified code, but oxc is for source code (whitespace more prevalent).

Identifier scanning: looks like the code in unicode-id-start is already optimal?

Internalizing minified identifiers - this is already done by string_cache, right? I didn't get what "Since JavaScript code is often minified, V8 uses a simple lookup table for single ASCII character strings" means.

Keywords and hashing - we already discussed this with regards to phf being optimized for binary space, not lookup speed.

Surrogate pairs seems like a small thing, possibly a hassle given we only work with a char iterator. But based on what's written, doesn't that mean read_string_literal is technically wrong since it doesn't allow a lone surrogate?

Overall, I don't see much actionable experiments here:

Stream rather read whole file to memory
keyword matching with perfect/faster hashing (known)

What other ideas do you see can try?

Identifier scanning:

I have a tiny micro optimization to make, I'll update unicode-id-start.

Keywords and hashing:

I'd like to try gperf but it seems Rust doesn't have this
so the next option would be to fork phf and add https://github.com/rust-phf/rust-phf/pull/236

Internalizing minified identifiers:

For oxc, I removed string-cache and went for `CompactString, this is better for cache locality I think.

Things I don't like so far:

this JSX testing branch, it's sitting on top of every read_next_token

https://github.com/Boshen/oxc/blob/b76ffb4826a4fbf871d020b35530add16760fbf6/crates/oxc_parser/src/lexer/mod.rs#L342-L344

this memmove here, and the drop comes with it. I'm not sure if there's any trick to keep this on the stack.

https://github.com/Boshen/oxc/blob/b76ffb4826a4fbf871d020b35530add16760fbf6/crates/oxc_parser/src/lexer/mod.rs#L180

this lookahead here and the code that comes with it (doing all operations on the heap). JavaScript lookahead is 4, but TypeScript lookahead seems infinite 😢

https://github.com/Boshen/oxc/blob/b76ffb4826a4fbf871d020b35530add16760fbf6/crates/oxc_parser/src/lexer/mod.rs#L95

I just looked at the code of V8's keyword matching to better understand what this is talking about:

Since the list of keywords is static, we can compute a perfect hash function that for each identifier gives us at most one candidate keyword. V8 uses gperf to compute this function. The result computes a hash from the length and first two identifier characters to find the single candidate keyword. We only compare the identifier with the keyword if the length of that keyword matches the input identifier length.

I think we should be able to do something similar get this working, for example use the phf with a faster hasher, and then just copy the generated table into our code?

I was quite interested in your discussions in #140. And I wrote a quick prototype of quick-lint-js's approach of matching keywords in rust: https://github.com/YangchenYe323/keyword-match-experiment
The idea is simple:

Generate an efficient collision-free hash table at build time:

Concatenate the first two characters (2 u8s), and last two characters (2 'u8's) of a string into a u32 as the hashed value. This gets us the selection of a key.
Hash the selection by doing some magic computation involving a random seed to get hash = f(selection, seed) Different values of seed is tested at build time to choose one without collisions.
Write the chosen seed and the corresponding hash table into a generated source file.

Simd string equality test: we can load the candidate str and keyword str into two Simd<u8, 16> since we can filter out longer candidates early on. But if the candidate token is actually less than 16 bytes, we have to read 16 bytes regardless, which makes this function unsafe. Then we compare those two Simd<u8, 16> and use a mask to extract the final result.

The benchmark result is quite promising:
The benchmark generates 8 random strings of random length between 2 and 11. And then match each of them against the js keywords using either a baseline match statement or hashing + simd. The hashing runs from 20% to 100% faster:

I ran this a couple times on an AWS ubuntu instance. The performance of the hashing approach is stably 32-34ns. The matching approach's run time varies from 33ns to as high as 100ns. Probably it's because the matching approach is more sensitive to the actual length of the string? That part is beyond me.
@Boshen It'd be great to get some insight from you. I could look into using this in oxc, too

Looks great!
You should definitely submit a PR imo
Few things I noticed:

I would test getting rid of

if clen < keyword_list::MIN_JS_KEYWORD_LENGTH || clen > keyword_list::MAX_JS_KEYWORD_LENGTH
        {
            return false;
        }

In practice, with the hashing approach it's probably not worth the branch misprediction. (vs hitting a n
2.
if len != candidate.len() {
            return false;
        }

This is repeated in is_keyword as well as simd_slice_match.
Only need one.
In quick-lint-js, this check is after the string comparison.
I assume @strager did his due diligence and found that it's overall faster to just do the string comparison than introduce branch misprediction.
Worth testing doing the same.

The SIMD comparison is currently possibly reading memory it shouldn't be reading, right? (because the slice length can be <16).

quick-lint-js pads the file contents right when it loads from disk, so reading 16 bytes is always safe.
I looked at this previously but we decided not to go through with it
So, I think the SIMD comparison has to be dropped.
Will be interesting to see how significant it is in practice.
I'd guess it varies, with TS codebases/files (which will have longer keywords more often) performing worse.

Should probably sprinkle #[inline]s.

re: the match approach fluctuating in benchmarks
Probably is due to string length having a large effect.
I think a fairer benchmark increases the sample size of tested strings, and comprises 50/50 of keywords and identifiers (I'm guessing. I don't know the actual distribution of keyword vs identifier in the wild)

@YoniFeng Thanks for tagging me!

I would test getting rid of [the early length check]

This would cause select to read out of bounds for 1-character variable names. https://github.com/YangchenYe323/keyword-match-experiment/blob/8032a59ff040406afc6eead92c8985a9dd241b30/src/hash.rs#L6-L18

I assume @strager's due diligence found that it's overall faster to just do the string comparison than introduce branch misprediction [from a length check].

This is correct.

Will be interesting to see how significant [SIMD] is in practice.

SIMD (or SWAR) provides a perf benefit on its own, but SIMD unlocks another optimization: branchless comparisons. SIMD+branchless improved performance significantly (at least 2x). (Going branchless for the whole algorithm, not just the string comparison, is even better.)
(Branchless string comparisons are not implemented in quick-lint-js right now. I didn't want to microoptimize quick-lint-js' current perfect hash table because I am going to throw it away eventually anyway. 😉 I mostly made the perfect hash table for a video (which I'll publish as soon as I make a YouTube thumbnail...).)
However, it seems that @YangchenYe323's experiment returns a boolean. I assume calling code is going to branch anyway, so making the string comparison branchless might not be worth it. (In quick-lint-js' case, keyword lookup returns an integer so things are more complicated.)

Maybe look at using https://docs.rs/cmov/latest/cmov/ to replace ifs?

How would you use cmov in the string comparison while avoiding out of bounds slice access?

and comprises 50/50 of keywords and random strings. (I'm guessing. I don't know the actual distribution of keyword vs identifier in the wild)

What I did for most of my benchmarking/profiling/optimizing is take word-looking things from jQuery and feed that into my lookup function: https://github.com/strager/perfect-hash-tables/blob/3938fb8f7ddb65d0a83c12bc4dcfeabaecc7c4fc/mixed.txt
This data set is about 20% keywords, 80% variable names (or words in comments). I didn't measure the ratio if you exclude comments.

And I wrote a quick prototype of quick-lint-js's approach of matching keywords in rust: https://github.com/YangchenYe323/keyword-match-experiment

Note that I own the copyright to quick-lint-js' code, and quick-lint-js is released with a GPL-3.0-or-newer license. If your project is effectively a copy-paste of quick-lint-js' code, please respect the software license.

@YangchenYe323 I think the following code is UB in Rust. You cannot grow a slice. I think Miri can catch this.
https://github.com/YangchenYe323/keyword-match-experiment/blob/8032a59ff040406afc6eead92c8985a9dd241b30/src/lib.rs#L57-L58

This would cause select to read out of bounds for 1-character variable names. https://github.com/YangchenYe323/keyword-match-experiment/blob/8032a59ff040406afc6eead92c8985a9dd241b30/src/hash.rs#L6-L18

I wasn't keeping track, I had your implementation in memory (AND with mask to avoid out of bounds).

I didn't want to microoptimize quick-lint-js' current perfect hash table because I am going to throw it away eventually anyway. 😉

Uhh - throw it away for what?

I mostly made the perfect hash table for a video (which I'll publish as soon as I make a YouTube thumbnail...).)
Wow https://github.com/strager/perfect-hash-tables seems fairly extensive/thorough.
Looking forward to it. (A video is enough? I'd think you need a blog series at least)

However, it seems that @YangchenYe323's experiment returns a boolean. I assume calling code is going to branch anyway, so making the string comparison branchless might not be worth it. (In quick-lint-js' case, keyword lookup returns an integer so things are more complicated.)

How would you use cmov in the string comparison while avoiding out of bounds slice access?

I didn't have the string comparison in mind, only the outermost layer.
But actually nvm, because if the method returns a boolean it's irrelevant.

Note that I own the copyright to quick-lint-js' code, and quick-lint-js is released with a GPL-3.0-or-newer license. If your project is effectively a copy-paste of quick-lint-js' code, please respect the software license.

Thanks for the reminder. I've added the same license to my repo.

@YangchenYe323 I think the following code is UB in Rust. You cannot grow a slice. I think Miri can catch this.
https://github.com/YangchenYe323/keyword-match-experiment/blob/8032a59ff040406afc6eead92c8985a9dd241b30/src/lib.rs#L57-L58

Yes it is UB, thanks for pointing this out! But we need to drop that part since we won't go for the Simd anyway.

I wasn't keeping track, I had your implementation in memory (AND with mask to avoid out of bounds).

I suppose we couldn't use a static mask either? because we don't pad the input so that 16 bytes are always readable. Could try mask against the actual length though.

Let's respect @strager's work and quick-lint-js's license.
@YangchenYe323 can you PR under quick-lint-js's license restrictions? I'll only merge when @strager says it's ok.

Let me check with @strager, I asked him in private.

@YangchenYe323 can you draft a PR with an implement with rust-phf and without SIMD? I know it is not perfect, but something slightly faster than the current string match implementation will be good enough for us right now.
We can't use the implementation from quick-lint-js's code given its license restrictions. And @strager will eventually teach us a clean-room implementation soon ;-)

@YangchenYe323 can you draft a PR with an implement using rust-phf and without SIMD? I know it is not perfect, but something slightly faster than the current string match implementation will be good enough for us right now.

Sure I'm on it. The tricky thing is that the baseline matching approach is hard to beat : ). I ran the benchmark using @strager 's JQuery input on (a). Rust HashMap, (b). Matching, (c). Perfect Hash:

Perfect hash is about 4x faster than using a rust HashMap (roughly consistent with the stats mentioned in #140 ) but only like 25% faster than just matching. Still need to see if we're able to get some speed up when this is migrated into oxc's lexer structure.

I did some more digging into the LLVM IR of the match_keyword function, and found that it's not simply comparison after comparison. Basically the input string would only compare with the match arms known to have equal length. The IR looks like this:

The big switch in bb5 branches corresponding to the length of the input string so actually on each invocation, at most ~15 memory comparison is done. Never are all 84 keywords compared. If we assume identifiers are much more common than keywords, these branches will all have a relatively low chance, and actually prediction could do pretty decently on this situation.
I suppose this might explain why #140 doesn't give the desired performance boost.

I did some more digging into the LLVM IR of the match_keyword function, and found that it's not simply comparison after comparison. Basically the input string would only compare with the match arms known to have equal length. The IR looks like this:
The big switch in bb5 branches corresponding to the length of the input string so actually on each invocation, at most ~15 memory comparison is done. Never are all 84 keywords compared. If we assume identifiers are much more common than keywords, these branches will all have a relatively low chance, and actually prediction could do pretty decently on this situation.
I suppose this might explain why #140 doesn't give the desired performance boost.

When I looked at the assembly, I was surprised to "find" it was doing comparison after comparison without first branching based on the length (I know this optimization exists in .NET - and would blindly assume LLVM has it as well), so I attributed it to "some reason I don't know about".
Nice to see you found this. I probably just wasn't thorough enough looking at the ASM.. (too lazy to go look again).
Makes complete sense.

Video on my perfect hash table implementation: https://www.youtube.com/watch?v=DMQ_HcNSOAI
Feel free to adopt ideas from this video.

After trying out a bunch of micro optimizations and by looking at the profiler at larger scale within the linter, I conclude that we don't need to look at the lexer anymore (for a long time).
The only remaining micro optimization is the phf keyword lookup method, which we'll leave it as an exercise after the video, unless https://github.com/Boshen/oxc/pull/171 gets an amazing result.


export PDF with extra background part

Preflight Checklist

[x] I agree to follow the Code of Conduct that this project adheres to.
[x] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Describe the bug
The crop option seems useless, I have tried export a PDF from pathes of "export as -> PDF" and "export as -> advanced" and both fail.
The exported PDF contains a whole page anyway.
To Reproduce
Steps to reproduce the behavior:

Go to 'File'
Click on 'Export as'
Select 'crop'
Click on 'Export'

Expected behavior
a PDF file only contains the diagram is exported
Screenshots
the currenet exported

draw.io version (In the Help->About menu of the draw.io editor):
v24.2.7
Desktop (please complete the following information):

OS: Windows 10
Browser Chrome
Browser Version  124.0.6367.60

I tested the problem in incognito/private mode with all browser extensions switched off, write "yes" below:
yes
Additional context
similar with https://github.com/jgraph/drawio/issues/4353 https://github.com/jgraph/drawio/issues/4348

Close all diagrams.net tabs, use clear.draw.io to push an update and test with 24.3.1.

And if I export PDF through "File -> print", it seems correct

Close all diagrams.net tabs, use clear.draw.io to push an update and test with 24.3.1.

Thanks, it works for the path "File -> Export as -> PDF". But still fail for "File -> Export as -> Advanced -> PDF", which I remember could work some days ago.

What does it show for the version under the help menu?

Close all diagrams.net tabs, use clear.draw.io to push an update and test with 24.3.1.

Thanks, it works for the path "File -> Export as -> PDF". But still fail for "File -> Export as -> Advanced -> PDF", which I remember could work some days ago.

Yes, it is not working now. The generated pdf are coming in multiple pages with blank background.

What does it show for the version under the help menu?

24.3.1

Working here too

Here, you can see, I am getting this one as the generated pdf. When I click on version 24.3.1 it's showing nothing.

We're going to add the zoom to the standard dialog and remove the advanced export to PDF option.

can we get an option to set a pdf background color?
in the older versions, the background in exported pdf's was the selected diagram background color, but atm its always white

I'm still facing this issue. I tried "File -> Export as -> PDF" and "File -> Export as -> Advanced -> PDF". They both failed.
I also experience the issue that the generated PDF will crop my large diagrams into multiple pages (all with large margins).
I tested on a desktop with Ubuntu 22.04 LTS, drawio v24.3.1, and a laptop with Windows 11, drawio v24.2.3. Anyone knows how to solve this?

I have the same problem...

I'm still facing this issue. I tried "File -> Export as -> PDF" and "File -> Export as -> Advanced -> PDF". They both failed.
I also experience the issue that the generated PDF will crop my large diagrams into multiple pages (all with large margins).
I tested on a desktop with Ubuntu 22.04 LTS, drawio v24.3.1, and a laptop with Windows 11, drawio v24.2.3. Anyone knows how to solve this?

I got a temporary solution for now.
First press "Ctrl + A" to select all the elements. Then "File -> Export as -> PDF". Then select both "Selection Only" and "Crop". This retains the previous "File -> Export as -> Advanced -> PDF" behavior.

I'm still facing this issue. I tried "File -> Export as -> PDF" and "File -> Export as -> Advanced -> PDF". They both failed.
I also experience the issue that the generated PDF will crop my large diagrams into multiple pages (all with large margins).
I tested on a desktop with Ubuntu 22.04 LTS, drawio v24.3.1, and a laptop with Windows 11, drawio v24.2.3. Anyone knows how to solve this?

I got a temporary solution for now.
First press "Ctrl + A" to select all the elements. Then "File -> Export as -> PDF". Then select both "Selection Only" and "Crop". This retains the previous "File -> Export as -> Advanced -> PDF" behavior.

It is the way I use usually but in my case does not work...

I also encountered the same problem. I chose to export as - Advanced - PDF, and the resulting PDF files were not on the same page, and there would be pagination. One solution is to select the content you need or select all, choose export as - pdf - check only the selected content & export a single page, but the margins of the pdf file obtained in this way are too small, I usually specify the margins as 5. This problem has not occurred before. I don’t know if it is a bug in the new version.

我仍然面临这个问题。我尝试了“文件 -> 导出为 -> PDF”和“文件 -> 导出为 -> 高级 -> PDF”。他们都失败了。
我还遇到这样的问题：生成的 PDF 会将我的大图表裁剪为多个页面（所有页面的边距都很大）。
我在装有 Ubuntu 22.04 LTS、drawio v24.3.1 的台式机和装有 Windows 11、drawio v24.2.3 的笔记本电脑上进行了测试。有人知道如何解决这个问题吗？

我现在有一个临时解决方案。
首先按“Ctrl + A”选择所有元素。然后“文件 -> 导出为 -> PDF”。然后选择“仅选择”和“裁剪”。这保留了之前的“文件 -> 导出为 -> 高级 -> PDF”行为。

像你一样，按照“文件 -> 导出为 -> 高级 -> PDF”的方式导出，之前我并没有出现这种问题，所以这应该是软件的bug吗？

我使用离线版本v24.2.5，按照“文件 -> 导出为 -> 高级 -> PDF”的方式导出，是没有问题的。

Fixed in v24.4.0 with the Crop option checked:

Use https://clear.diagrams.net/ to get the latest version.

If you find issues with 24.4.0, please create a new issue.


Add lane highlight to Vehicle Restrictions tool

When user is hovering a vehicle restriction icon, it is sometimes unclear which lane it corresponds to.

That was just an example from my current city; the more common scenario is huge roads with like 24 lanes where you have to start counting rows and lanes to work out what's being changed.

Could we pimp the relevant lane with a highlight to make it completely clear which lane is being altered?
The lane highlight would only be visible while the vehicle restriction icon is hovered.
Suggestion: White highlight (like outgoing lanes on Lane Connector tool) but without the border -a subtle highlight might be sufficient. Alternatively (might be overkill, try white first) the highlight could be coloured similar to the icon border (green or red) showing current state; that way if user hovers green icon the lane is green, they click to apply restriction = icon and lane turn red.

Refs: #33

@aubergine10 what is the difference between this and #33 ?
Also suggesting Shift modifier do the same thing as apply to all road toggle.

memory loss heh, totally forgot about the earlier issue until after I'd made this one but noticed it later on routine issue trawl and linked it in.

I manged to put the lane highlight on the top of segment highlight and stop them from blending with each other:

holding shift will turn the lane highlight to yellow.
Notice the yellow buttons: pressing those buttons while holding shift will apply changes to the whole lane.
I am planning to do the same with invert
pressing delete will activate the allow all button and show the 'button is pressed' sprite.

pressing shift+delete will allow all cars on the whole segment. the related GUI button will display yellow color and pressed sprite.

Observation:
Suppose you ban several vehicles on a single segment:

Then you shift click to ban another call on the whole lane:

You notice that all the other restrictions on the same lane have applied to the rest of the segments but restrictions from other lanes have not applied to the rest of the segments. strange ha?
I consider this to be a bug and I shall fix it :)

Yeah, the UI of this tool is a bit weird.
Suggestions:

Completely ditch the floating panel - it's confusing and largely obsolete I think?
Shift click an icon should only apply the changed state of that icon along the route; other vehicles should be unaffected

Example: If I shift+click car to restrict it (red icon state) then the car should be restricted along the route; other vehicle categories on the route would be unchanged.

Maybe have a Ctrl key shortcut that works a bit like speed tool - ie. toggles between lane-mode and direction-mode

@aubergine10

Completely ditch the floating panel - it's confusing and largely obsolete I think?

No I worked so hard to make it useful! I think we need to improve on it.
I think Priority road should also have a panel. in my opinion too many modifier keys with no proper visual feedback is confusing for users. Its better to have a panel that associates each button with a modifier key. if the button is pressed the particular mod is activated (until the button is pressed again). if the modifier key is pressed button texture changes to pressed and the particular mode is activated (until modifier key is released).

If we keep the floating panel, could we at least make it look a bit nicer?

Ditch invert button, seems pointless
Replace Ban/Allow all buttons with graphic buttons (similar to speed palette), rough mockup (still not found any remotely decent graphics apps for Windows):

Red with bar version of that icon to 'ban all'. Could be shortcuts for allow/ban (Ctrl+A, Ctrl+B) shown in tooltip of those buttons.
And maybe some button to toggle 'entire route' mode (tooltip would mention Shift key).

Yeah invert is pointless.
if shift (or road mode button) is pressed, lane highlights will become yellow and apply to road's length. the road mode button displays pressed sprite. related buttons show yellow color indicating they influence the whole road.

@aubergine10  I am thinking maybe we do not need this shift modifier at all. I don't think anyone will use it. "apply restriction to entire road" is good enough. we just need to get rid of the invert button.

I'd still keep Shift for 'apply to route' as that's consistent with all other tools and to me seems more intuitive than clicking a button on a panel. We can keep the button panel tool (it's looking a lot cleaner now and taking up less space) as some users will prefer that. But personally I'd prefer using shift just like in all other tools.

OK so I will remove the shift GUI button that I just added but the modifier remains.


update agent icon

update the icon for both the extension and the contributed dashboard tab.

Do we need the png file? I thought only svg..

Do we need the png file? I thought only svg..

PNG file is needed for the package.json, it only accept png file with 128 * 128 pixels


Locks don't stop Chest from being broken in my modpack.

I wish they had a spoiler option on this but here is a list of active mods. I don't see how any of these could be a problem that stops locks from functioning properly...unless it is already possible. I was in creative, put a lock on a chest, and had my friend try and break it (in survival). He couldn't open the chest, but he could break it...that, in my eyes is a problem.

  The point is not your friend cannot rob you, the point is you know if your friend robs you.
Additionally, letting people craft an unbreakable block is just too easy to abuse by griefers. Imagine your base surrounded by chests you cannot break. So no, I have no plans to make them unbreakable. If you have an idea to solve this problem without making it unbreakable I would be interested in hearing it.


Split glossary and acronyms into their own file

This would improve portability and readability when a lot of entries are needed. This would also allow the presentation template to share them.

Done. See acronyms.tex and glossaries.tex.


GET roles api not working

Current Behavior
Roles Api responds with 404 status.
GET: api/administration/user/app/${appId}/roles

Expected Behavior
API should respond with success(200) status
Steps To Reproduce
In app release process, when navigated to Technical Integration(step 4), the api call fails

the endpoint is replaced by https://github.com/eclipse-tractusx/portal-backend/issues/573

blocks https://github.com/eclipse-tractusx/portal-frontend/issues/686

@VPrasannaK94 after rechecking all scenarios we need one additional endpoint beside our new endpoint created as part of #573
Please also create the following:
For the app provider an endpoint is needed to fetch the roles of an app under the release process.
Till now we used an endpoint from administration service; but to ensure that we have a clear differentiation of the endpoints and services, we decided to create a own endpoint inside the app services.
/api/apps/AppReleaseProcess/agreementData
Endpoint: /api/AppReleaseProcess/{appID}/roles
Method: `GET``
Optional parameter: languagetag
Response Body
[
  {
    "role": "string",
    "descriptions": [
      {
        "languageCode": "string",
        "description": "string"
      }
    ]
  }
]

Validation:

only users assigned to the app provider company are allowed to call the endpoint details (user company ID = offers.provider_id)

Endpoint Summary: "The endpoint provides app providers an overview of configures app roles."