Add GitHub sorting options

Depends on #216 , click here for easy review
At the moment we take a very random sample of GitHub projects and mine these for patterns. A problem however with this approach is that a lot of the projects downloaded seem to be incomplete hobby projects, with very few files and next to no patterns.
It might therefore be necessary to at times limit ourselves to more relevant projects. To facilitate this the option to sort by stars and watchers has been added. Before, a single query was done to search for pom files on GitHub which contain a dependency on said library. Afterwards, the results were trimmed to the maxProjects before in turn querying GitHub for the projects which contain said pom files. Now however,  after the first query a query has to be done for each code file to get the stargazer and/or watcher count of the containing repository if we want to sort by either, making the process a lot slower.
These of course are optional arguments. It should be noted that downloading  takes a lot longer. However, seeing as in reality this will only be done every once in a while (not during every PR) it shouldn't be too big of a deal. For an emperical analysis of the tool this should be no problem at all. If speed is really of the essence then it is still possible to take a completely random sample as is the status quo.

Codecov Report

Merging #224 into master will increase coverage by 10.23%.
The diff coverage is 51.74%.

@@              Coverage Diff              @@
##             master     #224       +/-   ##
=============================================
+ Coverage     65.23%   75.46%   +10.23%
- Complexity      390      433       +43
=============================================
  Files            52       64       +12
  Lines          1162     1451      +289
  Branches        199      222       +23
=============================================
+ Hits            758     1095      +337
+ Misses          321      257       -64
- Partials         83       99       +16

Impacted Files
Coverage Δ
Complexity Δ

...n/org/cafejojo/schaapi/MissingArgumentException.kt
0% <0%> (ø)
0 <0> (?)

...haapi/miningpipeline/miner/github/SearchOptions.kt
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

.../cafejojo/schaapi/miningpipeline/MiningPipeline.kt
72.5% <100%> (+72.5%)
5 <3> (+5)
:arrow_up:

...fejojo/schaapi/GitHubMiningCommandLineInterface.kt
56.57% <56.57%> (ø)
0 <0> (?)

...otlin/org/cafejojo/schaapi/CommandLineInterface.kt
81.81% <65%> (+81.81%)
0 <0> (ø)
:arrow_down:

...ojo/schaapi/DirectoryMiningCommandLineInterface.kt
93.93% <93.93%> (ø)
2 <2> (?)

...jojo/schaapi/githubtestreporter/WebHookReceiver.kt
96.15% <0%> (ø)
4% <0%> (?)

.../cafejojo/schaapi/githubtestreporter/Properties.kt
55.55% <0%> (ø)
4% <0%> (?)

...fejojo/schaapi/githubtestreporter/CheckReporter.kt
83.33% <0%> (ø)
5% <0%> (?)

... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e72d84...389c673. Read the comment docs.


DPT training and inference script

Hi there,
I have checked the source code of DPT, but the GitHub link seems like it only has the OpenPrompt's code, while the DPT's code is not in the repo?

It is in the repo: https://github.com/Xianjun-Yang/DPT/tree/main/openprompt/prompts

Hi, I have checked the https://github.com/Xianjun-Yang/DPT/tree/main/openprompt/prompts, but I only title some  verbalizer.py. I am not sure if can you provide an easy tutorial/script/jupyter to utilize DPT to prompt tune a LLM's prompt~(e.g., T5).
I am very appreciative if you can provide a script to call the DPT's API.

all filename there containing 'adaptive' is used for dynamic prompting in DPT. You can try them.
I will make the scripts later.


Use rustc-hash instead of fxhash

Some data types in Rattler, like rattler_conda_types::RepoData, use FxHashMap instead of HashMap for better performance. This implementation utilizes fxhash, which employs a simpler hash function compared to the default one from std. fxhash was originally extracted from rustc, and since then, rustc-hash has been created and is maintained by rust-lang.
fxhash has not seen a release in seven years, and the last commit was five years ago. In contrast, rustc-hash is under active maintenance.
The author himself recommends using rustc-hash instead, as stated in https://github.com/cbreeden/fxhash/issues/10. As downstream users of this library have to interact with FxHashMaps, I think it would make sense to switch over to rustc-hash to avoid forcing people to depend on outdated crates and use the more popular crate instead.

Yeah makes sense! We might also consider using ahash though, I assume that will yield better performance.

But prs are welcome!


Internal errors when decoding TokenizedStrings with tokenizedstring.get

When using the following code:

I get the following output:

This is not helpful for the end user
An example here would be appreciated, as there is currently none, and the wiki is poorly documented.

The error is about as helpful as it can be because it says exactly what is wrong. It's attempting to concatenate bytestream[index:][:1], a tuple, to curr_bytes, a bytes object. Indeed, you're calling TokenizedString.get on the return of tokenizer.encode, which is a tuple of a bytes object and an OsVersion as stated in the docs (which is notably not the same as the wiki; the wiki is not intended to be library docs). You can see this error ahead of time from the output of print(tokenized).
But, outside of that error chain, how exactly did you derive this workflow? TokenizedString is an almost purely internal type, and never touted as anything more than that. .get is similarly a interface for the Python descriptor protocol, not a constructor. The type signature alone is often enough to guess what a method is doing; turning that last line into the functional tivars.TokenizedString.get(tokenized[0]) would just give you back "HELLO", which is certainly not what you want.
Are you trying to construct a TIProgram or TIString? examples/misc.py includes relevant code for constructing these types, which is supposed to be dead easy:
TIString("HELLO")

If you need to ensure tokenization for a particular model, you can do that too:
TIString.load_string("HELLO", model=TI_84PCET)


Hide children when type summary is active.

Align the logic of showing variable children with LLDB ObjectPrinter (link to the interesting line) used by "frame variable" command.
When type summary formatter is enabled, children should be hidden by default.
This can be overriden by not using lldb.eTypeOptionHideChildren so type summary can be used with synthetic provider as well (used for default STL providers inside LLDB).
Before - default Linux libstdc++ formatters bundled in LLDB:

After:

Makes easier debugging easier as you can spot easier that there is no reason to open the variable.

Thanks for the PR, but I still don't understand why...   Unlike the command-line interface, you can always not expand children of a variable in VSCode's tree view.   I personally prefer having the option to drill down into the raw representation.

Personally having the option to drill into a string creates a distraction as it is a leaf type and the summary contains all the information.
There is no need to dig deep, if the type summary is done correctly. Just from glance on the view you can see where it will make sense to drill deeper.
This also aligns CodeLLDB with other debuggers, namely ccptools GDB-MI debugger with pretty printing, that also doesn't show children from std::string.
For me it makes sense to drill down when the type is synthetic (nice [raw] view you have there) or there are actually children worth looking at.
But in the end, this is your project, so I am fine if you don't like this approach.
Optionally I could try to hide this behavior under some option for the debugger.

Even if I agreed to respect that flag and suppress string's children, I would immediately add them back under a [raw] node - for consistency with the rest of UI.   As such, I don't really see the point of doing it.
So, once again, thanks for the effort, but this isn't how I like my debugger to work.


attester: implement key id generation

Signed-off-by: Denis Semakin<EMAIL_ADDRESS>

@arturkow2000 Compiler warning about implicit function is fixed. Also  I do remember about your notes about using not RSA algo. This question is still open and isn't clear. Please review the other parts.

OMG Again...


S3-like local web server

Having a local web server that is S3-compatible is very nice for development. Having it embedded on Makes would be much appreciated and would allow other people to re-use it in their projects.
As a plus, adding options for populating buckets from existing S3 buckets can further save imperative code and turn it into declarative beauties
Example implementation:

https://gitlab.com/fluidattacks/product/-/tree/708bb9d1662b8431c2439d540465c72167b9b719/makes/applications/integrates/storage

I wanna work on this but its not totally clear for me, I would like to know the general functioning or objective of the example that you give us


google-colab 1.0.0 requires pandas==1.5.3, but you have pandas 2.0.3 which is incompatible

Getting Error while trying to use Google Colab.
I tried to install my pandas version, but seems didnt work. The same for tensorflow-metadata :(

issue has been resolved in latest update. no need to manually downgrade/upgrade anymore. the error part with pandas and google colab versions can be safely ignored.


build of the vue version

For what version of Nuxt UI are you asking this question?
v3.0.0-alpha.x
Description
Hi,
I'm exploring the Vue 3 compatibility section. When I run the following command:
pnpm run dev:prepare

I encounter this error:
ℹ Nuxt Icon client bundle consist of 99 icons with 47.35KB(uncompressed) in size                                                                                09:58:26
✔ Types generated in docs/.nuxt                                                                                09:58:27
ℹ Nuxt Icon server bundle mode is set to local                                                                                09:58:29
ℹ Running with compatibility version 4                                                                                09:58:29
✔ Types generated in devtools/.nuxt                                                                                09:58:30
failed to load config from C:\Users\chris\code\nuxt_ui\playground-vue\vite.config.ts
error during build:
TypeError [ERR_INVALID_URL_SCHEME]: The URL must be of scheme file
    at fileURLToPath (node:internal/url:1393:11)
    at fileURLToPath (file:///C:/Users/chris/code/nuxt_ui/node_modules/.pnpm/mlly@1.7.3/node_modules/mlly/dist/index.mjs:1971:25)
    at pathToFileURL (file:///C:/Users/chris/code/nuxt_ui/node_modules/.pnpm/mlly@1.7.3/node_modules/mlly/dist/index.mjs:1974:26)
    at _resolve (file:///C:/Users/chris/code/nuxt_ui/node_modules/.pnpm/mlly@1.7.3/node_modules/mlly/dist/index.mjs:2108:10)
    at resolveSync (file:///C:/Users/chris/code/nuxt_ui/node_modules/.pnpm/mlly@1.7.3/node_modules/mlly/dist/index.mjs:2111:10)
    at resolvePathSync (file:///C:/Users/chris/code/nuxt_ui/node_modules/.pnpm/mlly@1.7.3/node_modules/mlly/dist/index.mjs:2121:24)
    at NuxtEnvironmentPlugin (file:///C:/Users/chris/code/nuxt_ui/playground-vue/vite.config.ts.timestamp-1733302710251-85be7b7cfbb28.mjs:3303:20)
    at file:///C:/Users/chris/code/nuxt_ui/playground-vue/vite.config.ts.timestamp-1733302710251-85be7b7cfbb28.mjs:3339:5
    at file:///C:/Users/chris/code/nuxt_ui/node_modules/.pnpm/unplugin@1.16.0/node_modules/unplugin/dist/index.mjs:1800:32
    at file:///C:/Users/chris/code/nuxt_ui/playground-vue/vite.config.ts.timestamp-1733302710251-85be7b7cfbb28.mjs:3361:5
 ELIFECYCLE  Command failed with exit code 1.

Any guidance on how to resolve this issue would be greatly appreciated.

Do you encounter this when running dev:prepare in the nuxt/ui repository?

Yep, im in the v3 branch

For anyone experiencing this issue: initially, I was using the windows terminal, but after switching to WSL, it worked perfectly.


bug: segfault when performing dask sql queries

What happened:
Having upgrade to dask-sql-2022.4.1 I have been seeing a number of seg faults that appear to be related to the JPype java iterop.
What you expected to happen:
I was expecting dask-sql to be stable, but I require this upgraded version to support case insensitive queries with dask.
Minimal Complete Verifiable Example:
I am yet to repro with a concise example, but have included the seg fault log file if this helps.
---------------  S U M M A R Y ------------

Command Line: -ea --illegal-access=deny

Host: MacBookPro18,3 arm64 1 MHz, 10 cores, 32G, Darwin 21.4.0
Time: Fri Apr 29 20:24:41 2022 AEST elapsed time: 0.097347 seconds (0d 0h 0m 0s)

---------------  T H R E A D  ---------------

Current thread (0x0000000122c6c800):  JavaThread "Python Reference Queue" daemon [_thread_in_native, id=42243, stack(0x00000002aee70000,0x00000002af073000)]

Stack: [0x00000002aee70000,0x00000002af073000],  sp=0x00000002af072730,  free space=2057k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [python3.8+0x1bbacc]  collect+0x200
C  [python3.8+0x1bcf4c]  PyGC_Collect+0x9c
C  [_jpype.cpython-38-darwin.so+0x22368]  _ZN19JPGarbageCollection9triggeredEv+0x34
C  [_jpype.cpython-38-darwin.so+0x310c8]  Java_org_jpype_ref_JPypeReferenceNative_wake+0x18
j  org.jpype.ref.JPypeReferenceNative.wake()V+0
j  org.jpype.ref.JPypeReferenceQueue$Worker.run()V+42
j  java.lang.Thread.run()V+11<EMAIL_ADDRESS>v  ~StubRoutines::call_stub
V  [libjvm.dylib+0x329d14]  _ZN9JavaCalls11call_helperEP9JavaValueRK12methodHandleP17JavaCallArgumentsP6Thread+0x2e0
V  [libjvm.dylib+0x329074]  _ZN9JavaCalls12call_virtualEP9JavaValueP5KlassP6SymbolS5_P17JavaCallArgumentsP6Thread+0xec
V  [libjvm.dylib+0x32913c]  _ZN9JavaCalls12call_virtualEP9JavaValue6HandleP5KlassP6SymbolS6_P6Thread+0x64
V  [libjvm.dylib+0x3c0094]  _ZL12thread_entryP10JavaThreadP6Thread+0x78
V  [libjvm.dylib+0x6d55b8]  _ZN10JavaThread17thread_main_innerEv+0x80
V  [libjvm.dylib+0x6d53e4]  _ZN10JavaThread3runEv+0x17c
V  [libjvm.dylib+0x6d3104]  _ZN6Thread8call_runEv+0x78
V  [libjvm.dylib+0x57f96c]  _ZL19thread_native_entryP6Thread+0x13c
C  [libsystem_pthread.dylib+0x726c]  _pthread_start+0x94

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.jpype.ref.JPypeReferenceNative.wake()V+0
j  org.jpype.ref.JPypeReferenceQueue$Worker.run()V+42
j  java.lang.Thread.run()V+11<EMAIL_ADDRESS>v  ~StubRoutines::call_stub

siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 0x000000016b47b4b8

Register to memory mapping:

 x0=0x000000016b704030 is pointing into the stack for thread: 0x0000000112b83800
 x1=0x00000002af072880 is pointing into the stack for thread: 0x0000000122c6c800
 x2=0x00000002af072878 is pointing into the stack for thread: 0x0000000122c6c800
 x3=0x0 is NULL
 x4=0x0000000000006403 is an unknown value
 x5=0x0 is NULL
 x6=0x0000000000000001 is an unknown value
 x7=0x00000000004c4758 is an unknown value
 x8=

Anything else we need to know?:
This happens both locally on mac arm, and also in ubuntu linux build runners.
Environment:

dask-sql version:  dask-2022.4.0 dask-sql-2022.4.1 distributed-2022.4.0
Python version: 3.8.12
Operating System: mac osx
Install method (conda, pip, source):

The backtrace shows a referencing issue on the Python side.   Not sure which objects trigger it.  Clearly multithreaded, so perhaps reference handling issue in which a temporary ref counts dropped resulting incorrect collection and reuse of a live object.   Doesn't have to be JPype code itself, though it is certainly suspect.  If there is a clear method to simplify this and replicate it could be posted to the JPype issues, though these are often very difficult to trace down unless one can isolate what is happening at the time so we can isolate to the path.

Thanks for raising the issue! As @Thrameos mentioned, it can be difficult to track down the source of these kinds of errors without a reproducer. @brightsparc Would it be possible to also include the jpype, dask and distributed versions from an earlier environment where these segfaults weren't seen. Also are you using a LocalCluster for parallelizing the dask computations or is this just with a dask-sql context?


Add statistics of penalty shootouts

** Penalty Shootouts **
The penalty shootout data is available in the ./data/shootouts.csv and is loaded into the program.
Would be nice to add some graphs or statistics, such as:

win percentage on shootouts
is there a correlation between starting first and winning the shootouts
etc..

can i take this up?

For sure!


Any Way to Reverse the Order of Require.Context?

In my particular use case, I have a folder structure like this:

angular

directives

date-range

index.js

module.js

module.js

And id like to simply do
require.context('./angular');

Right now it seems to include files in the bundle in child first order. So my directives are getting put in the file before the module definitions. Essentially Id like files higher up in the hierarchy to appear before files deeper in the hierarchy.
Is there anyway to do this with require.context?

I don't think the order in which they are included in the bundle should really make a difference, should it? I'm guessing you're concerned about the order in which they are executed if you have implicit dependencies like
// a.js
window.ModuleA = ...;

// b.js
window.ModuleB = /* using window.ModuleA */;
```

In that case, I think you can achieve it like this:

```js
var context = require.context('./angular');
var modules = context.keys();

// sort them
var sortedModules = modules.slice().sort(function(a, b) {
  // by directory depth
  var diff = a.match(/\//g).length - b.match(/\//g).length;
  if ( diff !== 0) return diff;

  // if depth is the same, sort alphabetically to make it stable
  return a < b ? -1 : (a > b ? 1 : 0);
});

// execute them
sortedModules.forEach(function(key) {
  context(key);
});
```

For this use case, the module files create the modules. The directive files then add directives to those modules. If those modules aren't run and exist before the directive code is run, things won't work right. These aren't commonjs at the moment. I'm really just using webpack process (es6 babel etc) and concat all the files together.
So I started comparing the output for better understanding, and it seems I totally misunderstood what's going on. I haven't gotten to the point yet of testing the bundle on a page and had assumed that the order in which the modules appeared in output was the order they would be executed (aka a pure concat). WRONG. I didn't realize an array of module functions was being passed to the bootstrapper. This is commonjs for front end. Nothing is run until you require a module. All require.context was doing was setting up the modules. So yes, the order of modules passed to the bootstrapper doesn't matter. The order in which you require those modules does, hence your example of re-ordering the keys based on directory depth and then executing them in that new order.
Thanks for taking the time to answer my question.


shortlink is not parsed on some sites

Is not getting the ready shortlink tag on many pages, ex :
https://www.pipiscrew.com/2021/08/jj-still/
https://www.makeleio.gr/?p=3881024
but on this, is working fine
https://pmig96.wordpress.com/2021/04/29/palmos-on-raspberry-pi/

I execute the code of data/js/discover/default.js on console and working fine (result the shortlink), possible somewhere before stumble.

Thank you for filing this - I will check the pages you are talking about. It is possible that something is messing with the message passing between the page and the add-on :o
I appreciate your report!

I re-confirmed that this URL https://www.makeleio.gr/?p=3881024 works correctly. I wonder if this was a bug in the Mozilla add-ons framework that now went away. I will close this for now, but please reopen if you encounter more issues!


[Suggeston] Include the 6 decimal places when using Mrai as units, so more precise amount can be sent

Would be more convenient / useful as currently the only way to send precise amounts is to use rai as units. Possibly more useful if value of Mrai increases significantly.
i.e it is not possible to send 1000.543212 Mrai if Mrai are the default units, only 1000 Mrai can be sent.
Similar suggestion for krai too, include the 3dp.
Or just have it available as an option which can be turned off if people don't like the decimals.
But being able to send precise amounts using Mrai as default units is a must, in my opinion.

Closing out as duplicate of https://github.com/nanocurrency/raiblocks/pull/1301


Add support for modals with forced scrollbars (overflow-y: scroll)

Scrollbar showing/hiding and the related "layout jumping" is usually percieved pretty negatively by users. This is less issue on static web pages, however with dynamic pages or single-page web applications (e.g. AngularJS based) it becames very uncomfortable and distracting.
To workaround this a standard solution is to force browsers to always show vertical scrollbar by setting overflow-y: scroll on <body> element. However this is incompatible with Bootstrap's modal dialog as it checks for scrollbar presence only by checking that the content height is bigger than the viewport.
I would like Bootstrap to support forced scrollbars (i.e. change the detection when the scrollbars are shown).
JS-Bin to demonstrate the current state of things: http://jsbin.com/johobubafa/1/

Hi @pavelhoral!
You appear to have posted a live example (http://jsbin.com/johobubafa/1/edit), which is always a good first step. However, according to Bootlint, your example has some Bootstrap usage errors, which might potentially be causing your issue:

W002: <head> is missing X-UA-Compatible <meta> tag that disables old IE compatibility modes
W003: <head> is missing viewport <meta> tag that enables responsiveness
line 18, column 5: W007: Found one or more <button>s missing a type attribute.

You'll need to fix these errors and post a revised example before we can proceed further.
Thanks!
(Please note that this is a fully automated comment.)

Can I ignore the automated comment above? Mentioned validation errors have nothing to do with the feature request :).

Forked the JSBin with the 3 fixes for @twbs-lmvtfy : http://jsbin.com/cuzuqibihu/1/edit?html,output

@pavelhoral This has supposedly been fixed in current master (see https://github.com/twbs/bootstrap/pull/15746#issuecomment-76573805 ). Can you re-test and confirm? Thanks.

I confirm that it works with the current master (tested with FF 36 and Chrome 40 on Kubuntu). Thank you!


Q: How to add custom taxonomies to a specific menu as sub menu

Wordpress Version : 4.4.2
Timber Version: 0.22.5
I have a custom taxonomy (victim groups) for a custom post type(victim).
I have a menu entry for victims and want to get a sub menu with the different victim groups. How can i add them programaticly?
cheers and thanks in advance
bambamboole

Hi @bambamboole - it may worth posting this on our stack overflow page as you may get better support for this question there.
http://stackoverflow.com/questions/tagged/timber

ok thx i'll give it a try!


Links are almost invisible for macOS theme in yellow on white

Using the current version of Joplin (2.10.10 (prod, darwin)) and the current version of the macOS theme all links show up in yellow on white which makes them almost invisible. The links are blue without the theme. I get the yellow on white for both Ventura and my old High Sierra.

Hi @beatrixwillius , it's a bit unfortunate but not all themes offered by Codemirror (the Markdown editor that Joplin is using under the hood) are optimised for contrast. The light theme you're using 3024-light is one of them. You can however easily select another theme under Joplin settings › macOS theme › Markdown editor light (needs a restart after applying). Does that solve your issue?

Thanks, that did the trick!


Scan components to find asset folders to serve as static resources

Also:

Updates the readme to mention the service_path
Updates the route static tests to use tap, which has a compatible API, but provides better async support than tape
Updates tests

To expand upon my change of tape to tap: tape didn't have great async support, whereas tap does. While I think there are even better test packages out there, using tap is fairly unobtrusive as it has a compatible API, and the test command doesn't even need modifying. All that changes is the require statement at the top of the file...we can do this incrementally. Tests can go from:
test('When a file that does not exist within a static route is requested', t => {
  const app = callRoutesStatic()

  request(app)
    .get('/static.txt')
    .end((err, res) => {
      t.equals(err, null, 'it should not invoke an error')
      t.equals(res.status, 404, 'it should return 404')
      t.end()
    })
})

To this:
test('When a file that does not exist within a static route is requested', async t => {
  const app = await callRoutesStatic()
  const {error, status} = await request(app).get('/static.txt')
  t.ok(error, null, 'it should invoke an error')
  t.equals(status, 404, 'it should return 404')
})

Which is better IMO.
Also converted the method signature to use objects rather than function(arg1, arg2, arg3) etc. As I think it's more readable...ideally though, if we agree, we can add such rules to a linter at some point!

@solidgoldpig setting this as a draft PR since this is my first. Let me know of any feedback!

Also heads up: the tests were failing for me in master and caused a bunch of confusion as a different branch (the refinement branch) had some fixes...would be good to keep master clean - having a proper PR process, where our PRs are continuously tested for linting errors, tests failures etc. and then rejected if necessary would be a good idea to do!

Code review feedback so far:

Fix linting
Don't use experimental Node.js features for the async stuff, but rather use Util.promisify

Hey @solidgoldpig thanks for looking at this! Have pulled in your changes, resolved conflicts, addressed code review feedback and now using Util.promisify instead of experimental Node.js features. If you want to have another look that'd be useful.
Also if you think it's worth doing the ava stuff in this branch, can do that, otherwise will save it for next time

I think leave the ava-ification for another (but soonish) time
The introduction of tap I see has now triggered a bunch of audit warnings.
Locally I ran npm audit fix and it sorted 8 out of the 9 issues, but tap's handlebars dependency is still complaining
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ handlebars                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.0.14 <4.1.0 || >=4.1.2                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ tap [dev]                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ tap > nyc > istanbul-reports > handlebars                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/755                       │
└───────────────┴──────────────────────────────────────────────────────────────┘

For that particular remaining advisory, fortunately we are immune since the attack vector is from user supplied input. In our case, tap is only used as a dev dependency and is not exposed to the user.
Also, looking at this: https://github.com/wycats/handlebars.js/issues/1495#issue-407574076 they have fixed the issue. The version they list as fixed was the same version we had installed, I assume the NPM advisory hasn't been updated to take into account backported fixes.
I updated the package lock file anyway though

Update: Just to be certain, I verified 4.1.0 (which we would have installed):

Includes the security fix
Has been published on NPM

Good to go


sway-parse only lexes comments, and doesn't give spans

Blocker for handling comments in swayfmt v2
At some point we will need to find a way to handle comments so that we get types for comment & multiline comment as well as the spans. We'll need this if we want to have reformatting features for comments like: wrap comments or for reformatting multiline comments to normal comments.

Hmmm I don't remember having trouble with the new parser and comments. Can you show specifically what's not working?

Hmmm I don't remember having trouble with the new parser and comments. Can you show specifically what's not working?

It's not that it's not working per-say, rather that it's just not there at all.
In order to reformat comments, we need handling of comments from the parser. The only thing I can find is an error UnclosedMultilineComment, which is a variant of token::LexErrorKind.

To be specific we would just need the type of comment, multiline or not, and the span.

NVM, found the function token::lex() that handling comments, but it's handling comments in a similar way that the current formatter handles them. I was under the impression we would get comment handling as items from the ast.

Closing this since my initial statement was incorrect, and will bikeshed w/ tooling on how to proceed when the time comes. Thanks @mohammadfawaz !

@sezna @mitchmindtree for more insight if you can provide it!

I don't mind taking this on at some point as well, just trying to get more information on this thread.

I was under the impression we would get comment handling as items from the ast.

@eureka-cpu just to clarify, note that there's a couple of "parse" phases in sway:

The sway-parse crate's parse_file function, which parses a string assumed to represent the source of a Module.
The sway-core crate's parse function, which calls sway_parse::parse_file internally but also transforms the sway_parse::Modules into sway_core::ParseModules which are a little more minimal and are stripped of tokens etc that are unnecessary for the following type-checking phase. This parse phase should probably be called parse_to_untyped_ast or a better name along those lines.

In the case of the formatter, it's likely best to use sway-parse directly as it's the closest mapping to the direct source.
W.r.t. retrieving the comments, I can understand why we might not want to include them in the parsed tree, as comments can go just about anywhere and it would likely add a lot of noise to the structure of the tree.
I wonder if instead it would be OK to add a Comment variant to the TokenTree type so that they can at least be emitted via token::lex? This way, perhaps the formatter could first lex the file and use the TokenStream to create a map of spans to comments, which we could then provide to the format implementations?
cc @canndrew might have some thoughts/wisdom on how to best expose comments.

I wonder if instead it would be OK to add a Comment variant to the TokenTree type so that they can at least be emitted via token::lex?

That would certainly be better than trying to somehow include them in the AST but it's still not ideal. It means that any parsing routine that consumes tokens from a TokenStream would have to skip over comments.
Maybe a better option would be to have CommentedTokenStream and CommentedTokenTree types, make token::lex return a CommentedTokenStream, and have a CommentedTokenStream::strip_comments method that returns a TokenStream. How useful would that be for swayfmt @eureka-cpu?


Support github.com/breml/tfreveal

https://github.com/breml/tfreveal

Thank you.

#27030

v4.222.0 is out 🎉
https://github.com/aquaproj/aqua-registry/releases/tag/v4.222.0


Changes for running LLDB test suite for Swift on PowerPC64LE

The attached changes are required for running LLDB test suite related test cases for Apple Swift 5 on PowerPC64LE.

Hi @lilyball @penryu @nico @ismail can you please review this?

Hi @lilyball @penryu @nico @ismail can you please review this?

Also, please let me know the correct branch to submit these PR changes in case this branch is not the correct one. Thanks in advance.

Wrong nico?

I don't normally review this area of code, but these changes look fine.  Merging to stable is fine, it will automerge to branch upstream-with-swift.

Wrong lily? Perhaps you meant @millenomi?

@dcci can you please review and approve this if it looks OK?

@swift-ci test

The change in SwiftLanguageRuntime.cpp looks fine. One between Saleem and Stefan should take a look at the changes in CMake before this goes in.

I know Stefan is OOO so @compnerd is the best person to review this.

@swift-ci please test Linux platform

@swift-ci please Linux platform

@swift-ci please test Linux platform

@compnerd @swift-ci any updates on this?

@swift-ci please test Linux platform

@sarveshtamba sorry for the delay. This should be unblocked now.

@swift-ci test and merge


Optimize mono runtime

We need to figure out what parts of mono.wasm and mono.asm.js we are using and remove everything else.
The original approach is described below.

In your wasm binary, implement a function that takes an i32 and, when called, passes it through to
JS code. On the JS side, track which numbers you receive.
Convert the original .wasm file to .wast using 'wasm2wat' from https://github.com/WebAssembly/wabt
Use the annotateWast function to emit an instrumented version of that .wast file. Each function
in it will be modified so that it starts by invoking function 115, passing the ID of the caller.
Change the '115' const to whatever else is the ID of your compiled function from step 1.
Convert the instrumented .wast back to .wasm using 'wat2wasm'
Run your application using the instrumented .wasm file. After exercising all reachable code paths,
use the browser debug console to evaluate the list of function IDs that were invoked.
Update the 'usedFunctionIds' array above.
Use the stripWast function to generate a stripped version of your original .wast file in which all
the uncalled functions are replaced with empty stubs.
Convert the stripped .wast file to .wasm using wat2wasm
Use the stripped .wasm file in your application

I was thinking about giving this a try, but I'm not sure what After exercising all reachable code paths entails.  I also couldn't find an annotateWast function referenced anywhere, but I think I might be able to get decent results out of Wasabi.

@danroth @stevesandersonms

@drewdelano Thanks for your interest.
We don't yet have a final plan for how to know what to strip out from mono.wasm. The most realistic approach is probably to find ways of understanding what's in mono.wasm and making case-by-case decisions about what could be removed. For example, if large parts of it deal with low-level APIs for TCP listeners, we could remove them, because web apps can't listen for direct incoming TCP connections.
The approach mentioned by @javiercn above is really a description of a proof-of-concept form of optimization I did a while ago. It's not a totally realistic approach, since in reality you can't enumerate every possible code path. If you have any clever ideas about how this could be make to work reliably that would definitely be interesting.
If you have time to investigate things in this area, it would be great to find out what we can about how the payload size of mono.wasm breaks down into different bits of functionality so we could reason about what might be viable to remove. If you find that tools like Wasabi provide good insights, or even any info at all, it would be great to know about that!

@SteveSandersonMS I'll take a look at it this weekend.
The two approaches I see are:

Instrument mono.wasm and set the pre-call hook to batch and then report which functions are called.  Then merge this change into master and wait for better data.  The downside is that then there needs to be some sort of server collection point and it seems a little sketchy from a privacy point of view.

Same basic concept as above, but with a local collection point per site.  The idea being the debug build would always have the full mono.wasm, but release build mode would trim mono.wasm to only the functions that were hit while in debug mode.  The downside is that if you miss something in local testing it would break in release mode and if you stopped using something you'd need some way to clear the local cache of used functions or it would always include some functions you weren't using.

That's great to hear. I'll be interested in what you find!
As for making optimisations that could be built into the product, we'd probably need to be able to reason about why a particular thing is legit to strip out (as in, why we believe it's never going to be used in anyone's app, e.g., because it's some code specific to running on Win32 or macOS which isn't applicable on WebAssembly). Observing that something happens not to be used during a particular test pass is interesting and can guide us towards knowing what to remove, but on its own wouldn't be enough of a guarantee that it's safe to remove in all cases.

Wasabi didn't work, sadly.  It just results in an "Aw, Snap!" page in Chrome.
I moved on from there to a more manual approach:

Use wasm2wat to dump mono.wasm to text so I could edit it
Add a custom function (currently 5424th which comes up later):

  (func (;instr;) (type 1) (param i32)
    i32.const 134211724
    i32.const 0xdeadbeef ;; [134211724] = 0xdeadbeef
    i32.store

    i32.const 134211728
    get_local 0
    i32.add
    i32.const -1
    i32.store8    ;; [134211728 + funcNum] = 0xff
  )

This is a very hacky function that always writes a breadcrumb (0xdeadbeef) + 0xff at offset just past our breadcrumb + the function number offset.  If a function WAS called then there should be 0xff at it's offset; otherwise, it should be 0.  There's probably a way better way to do this, but I was having difficulty understanding the WASM memory model.

I wrote a LINQPad script to instrument all of the functions:

var lines = File.ReadAllLines(@"PATH_TO_PROJECT\FlightFinder\FlightFinder.Client\bin\Debug\netstandard2.0\dist\_framework\wasm\mono.txt").ToList();
for(var i = 0; i < lines.Count;i++) {
	var line = lines[i];
	if (line.TrimStart(' ').StartsWith("(func (;") && !line.Contains(";instr;")) {
		var funcNumStr = line.Split(';')[1];
		var funcNum = int.Parse(funcNumStr);
		var targetLine = i + 1;
		while(lines[targetLine].Contains("(local ")) {
			targetLine++;
		}
		lines.InsertRange(targetLine, new[] {
			"i32.const "+funcNumStr,
			"call 5425"  //********* ...I said it would come up later
		});
	}
}
<EMAIL_ADDRESS>lines);

This adds two instructions to each function: a const of the function offset + a call to our instrumentation function

After adding the function and running the LINQ Pad script, I used wat2wasm to turn it back into a binary and tested to make sure nothing broke
I then needed some way to see which functions had executed, so I added this to mono.js:

function dumpUsage() {
    var arr = new Uint8Array(Module.wasmMemory.buffer);

    for(var i = 0; i < arr.length; i++) {
        if(arr[i] == 0xde && arr[i - 1] == 0xad && arr[i - 2] == 0xbe && arr[i - 3] == 0xef) {
            console.log("found deadbeef at: " + i);
            break;
        }
    }
    console.log("done looking...");

    var whereDeadbeefWasFound = 134211727;
    arr = arr.subarray(whereDeadbeefWasFound + 4, whereDeadbeefWasFound + 4 + 5424);

    var funcsCovered = arr.filter(function(e) { return e == 255 }).length;
    var badData = arr.filter(function(e) { return e != 255 && e != 0 }).length;
    console.log("funcs covered: " + funcsCovered);
    console.log("percent covered: " + funcsCovered / arr.length * 100);
    console.log("bad data: " + badData);
}

This is also pretty hacky.  I basically scan through the entire memory space available to the Blazor app (around 128 MB worth) until I find the deadbeef breadcrumb and write that out.  Once I had that I update the mono.js again to set whereDeadbeefWasFound to the number the function executed on the first step.  This could be refactored to make it much better, but it was good enough to get some raw metrics out of and surprisingly fast (less than a second execution time for dumpUsage on my 4 year old laptop).
All of this yields some pretty high level stats, but I plan to drill into them more next weekend:

That's really interesting. Thanks for the update, @drewdelano!

any idea if there's anything like symbol information for the mono binaries?

@kumpera would be the best person to ask.

For anyone else trying to perform similar steps, I'm following a lot of the steps contained in the "How to upgrade mono" docs: https://github.com/aspnet/Blazor/blob/master/src/mono/HowToUpgradeMono.md
I had to add  -s ERROR_ON_UNDEFINED_SYMBOLS=0 to mono.csproj's <CommonEmccArgs> node to get things building
Then I ran across two other interesting switches you can add to the CommonEmccArgs from emcc --help:
* "-g4": Preserve LLVM debug information. This is the highest
  level of debuggability. If "-g" was used when compiling the
  C/C++ sources, this shows line number debug comments, and
  generates source maps.

   Note:

     * This debugging level may make compilation at
       optimization level -O1 and above significantly slower,
       because JavaScript optimization will be limited to one
       core (the default in "-O0").

     * Source maps allow you to view and debug the *C/C++
       source code* in your browser's debugger! This works in
       Firefox, Chrome and Safari.

"--emit-symbol-map"
   Save a map file between the minified global names and the original
   function names. This allows you, for example, to reconstruct
   meaningful stack traces.

   Note: This is only relevant when *minifying* global names, which
     happens in "-O2" and above, and when no "-g" option was specified
     to prevent minification.

But they don't appear to be compatible with each other

LINQPad script to turn symbols + wat instructions into something usable:

we want both the function offset + number of instructions so we can sort them later
var watFile = @"C:\Users\Drew\Desktop\Blazor\src\mono\dist\optimized\wasm\mono.txt";

var lines = File.ReadAllLines(watFile).ToList();
var insts = new Dictionary<string, int>();
var paranCount = 0;
var lastFunc = "";
for(var i = 0; i < lines.Count; i++) {
	var line = lines[i];
	// look for the start of methods
	if (line.TrimStart(' ').StartsWith("(func (;") && !line.Contains(";instr;")) {
		paranCount = (line.Split('(').Length - 1) - (line.Split(')').Length - 1);
		lastFunc = line.Split(';')[1];
		insts.Add(lastFunc , 0);
	} else if(paranCount != 0) {
		paranCount += (line.Split('(').Length - 1) - (line.Split(')').Length - 1);
		insts[lastFunc]++;
	}
}

var syms = @"C:\Users\Drew\Desktop\Blazor\src\mono\dist\optimized\wasm\mono.js.symbols";
var lines2 = File.ReadAllLines(syms);
File.WriteAllLines(syms, lines2
	.Select((l, idx) => /* see foot note */ insts.ContainsKey(idx.ToString()) ? "{ \"name\": \"" + l.Split(':')[1] + "\", \"insts\": " + insts[idx.ToString()] + " }," : null)
	.Where(m => m != null)
	.ToList());

// foot note: import funcs also count as functions which means the first 140 or so functions don't have an implementation, so skip them as they aren't instrumented anyway

.
Updated mono.js to easily sort unused functions by their weight:

Configure to show only the top 300 as that's still probably more than we can comfortably work with
function dumpUsage() {
    var arr = new Uint8Array(Module.wasmMemory.buffer);

    for(var i = 0; i < arr.length; i++) {
        if(arr[i] == 0xde && arr[i - 1] == 0xad && arr[i - 2] == 0xbe && arr[i - 3] == 0xef) {
            console.log("found deadbeef at: " + i);
            break;
        }
    }
    console.log("done looking...");

    var whereDeadbeefWasFound = 134211727;
    arr = arr.subarray(whereDeadbeefWasFound + 4, whereDeadbeefWasFound + 4 + 5568); // 4 to skip past the breadcrumb, 5568 the number of functions to potentially cover

    var funcsCoveredCnt = arr.filter(function(e) { return e == 255 }).length;
    var badData = arr.filter(function(e) { return e != 255 && e != 0 }).length;
    console.log("funcs covered: " + funcsCoveredCnt);
    console.log("percent covered: " + funcsCoveredCnt / arr.length * 100);
    console.log("bad data: " + badData);

    var uncoveredFuncs = [];
    var j = 0;
    while (j < dumpUsageLookup.length - 1) {
        if (arr[j] == 0) {
           uncoveredFuncs.push(dumpUsageLookup[j]);
        }
        j++;
    }
    var sortedFuncs = uncoveredFuncs
        .sort((a, b) => a.insts == b.insts ? 0 : a.insts > b.insts ? -1 : 1)
        .map(m => m.name + " (" + m.insts + ")")
        .slice(0, 300);

    console.log("Uncovered funcs:");
    console.log(sortedFuncs.join("\n"));
}

var dumpUsageLookup = [
    { "name": "__growWasmMemory", "insts": 4 },
{ "name": "stackAlloc", "insts": 16 },
...
];

.

This is done by introducing a "enable-minimal" feature and use it through the code to disable the codepaths we don't want.
I'll cook up a sample PR that shows how to do this sort of work on mono.

Hmm, process is not a good case for enable-minimal, but to replace the unix port with a wasm specific.

Here's a skeleton commit that sets up the stage to remove a lot of the non-sense platform stuff like process/socket support. https://github.com/kumpera/mono/commit/5a4fcf5e27a5456790b2febab1f0c6fe2c5b8e80
It still requires some work

I was going to start by testing this concept locally by just removing the instructions for those functions in the wat file and then rebuilding it.  Just to test that all of this is actually working.
But I agree changing it in mono is probably the best long term goal.

I got distracted with some other stuff.  I should be returning to this next weekend.


No data transmitted after latest firmware update

After the last iOS App update came a firmware update for the flower power sensor. Since then, the node-flower-bridge still connects fine with the device, but no data is transferred to the cloud service.

I'm seeing the same issue. I see data getting synced to https://myflowerpower.parrot.com but it isn't showing up in the iOS app.

The v2 branch seems to work.

Tried v2 and copied over my credentials.json from the "v1" version, but i get "Error: Invalid URI "undefined/user/v3/authenticate"". Has the API url changed somehow?

when i add the url: 'https://api-flower-power-pot.parrot.com' to the credentials.json, i get the following error:
url: 'https://api-flower-power-pot.parrot.com/user/v3/authenticate',
method: 'POST' }
CODE: 400
error: invalid_client

Oh, you may need a new api key. I was getting an invalid client error after making sure everything else was there, and getting a new key made it work.
Here's where to request a key:
https://api-flower-power-pot.parrot.com/api_access/signup
Here are the official docs for the new/current API:
http://developer.parrot.com/docs/FlowerPower/
Here's my output when I run, with my own data replaced with XXXX:
pi@raspberrypi:~/node-flower-bridge $ ./bridge display
-- Flower Bridge --
[---] No program is running
node /home/pi/node-flower-bridge/start.js
POST/urlencoded /user/v3/authenticate
client_id: XXXX
client_secret: XXXX
username: XXXX
password: XXXX
grant_type: password
app_identifier:
{ headers:
{ 'Content-Type': 'application/x-www-form-urlencoded',
Authorization: '' },
body: 'client_id=XXXX&client_secret=XXXX&username=XXXX&password=XXXX&grant_type=password&app_identifier=',
url: 'https://api-flower-power-pot.parrot.com/user/v3/authenticate',
method: 'POST' }
✔ Login!
and then a bunch more stuff.

the new api key worked, thank you.
strangely the v2 version finds only one of two sensors (it finds 2, but can only connect to one) and even from the one that he has connected to, no new data is transmitted to the cloud.
It seems to me that this is broken and as far as i can see, there is no active development anymore for the bridge ... but hey, it was fun while it lasted.

After patching (unknown, why the node package is not updated) the url in the file /build/node-flower-bridge/node_modules/flower-power-api/FlowerPowerCloud.js
FlowerPowerCloud.url = 'https://api-flower-power-pot.parrot.com';
and
const DEBUG = true;
I get the message
pi@raspberrypi:/build/node-flower-bridge# node start.js
POST/urlencoded /user/v2/authenticate
client_id: xxxx
client_secret: xxxx
username: xxxx
password: xxxx
grant_type: password
{ headers:
   { 'Content-Type': 'application/x-www-form-urlencoded',
     Authorization: '' },
  body: 'client_id=xxxx&client_secret=xxxx&username=xxxx&password=xxxx&grant_type=password',
  url: 'https://api-flower-power-pot.parrot.com/user/v2/authenticate',
  method: 'POST' }
✔ Login!
[Nov 06 2016 14:58:43]: New process every 15 minutes
GET/json /sensor_data/v4/garden_locations_status
{ headers:
   { 'Content-Type': 'application/json',
     Authorization: 'Bearer xxxx' },
  body: 'null',
  url: 'https://api-flower-power-pot.parrot.com/sensor_data/v4/garden_locations_status',
  method: 'GET' }
GET/json /sensor_data/v3/sync
{ headers:
   { 'Content-Type': 'application/json',
     Authorization: 'Bearer xxxx' },
  body: 'null',
  url: 'https://api-flower-power-pot.parrot.com/sensor_data/v3/sync',
  method: 'GET' }
GET/json /user/v4/profile
{ headers:
   { 'Content-Type': 'application/json',
     Authorization: 'Bearer xxxx' },
  body: 'null',
  url: 'https://api-flower-power-pot.parrot.com/user/v4/profile',
  method: 'GET' }
[Nov 06 2016 14:58:43]: Error in getInformationsCloud

There seems to be a problem with https://api-flower-power-pot.parrot.com/user/v4/profile

After patching (unknown, why the node package is not updated) the url in the file /build/node-flower-bridge/node_modules/flower-power-api/FlowerPowerCloud.js
FlowerPowerCloud.url = 'https://api-flower-power-pot.parrot.com';
and
const DEBUG = true;
I get the message
pi@raspberrypi:/build/node-flower-bridge# node start.js
POST/urlencoded /user/v2/authenticate
client_id: xxxx
client_secret: xxxx
username: xxxx
password: xxxx
grant_type: password
{ headers:
   { 'Content-Type': 'application/x-www-form-urlencoded',
     Authorization: '' },
  body: 'client_id=xxxx&client_secret=xxxx&username=xxxx&password=xxxx&grant_type=password',
  url: 'https://api-flower-power-pot.parrot.com/user/v2/authenticate',
  method: 'POST' }
✔ Login!
[Nov 06 2016 14:58:43]: New process every 15 minutes
GET/json /sensor_data/v4/garden_locations_status
{ headers:
   { 'Content-Type': 'application/json',
     Authorization: 'Bearer xxxx' },
  body: 'null',
  url: 'https://api-flower-power-pot.parrot.com/sensor_data/v4/garden_locations_status',
  method: 'GET' }
GET/json /sensor_data/v3/sync
{ headers:
   { 'Content-Type': 'application/json',
     Authorization: 'Bearer xxxx' },
  body: 'null',
  url: 'https://api-flower-power-pot.parrot.com/sensor_data/v3/sync',
  method: 'GET' }
GET/json /user/v4/profile
{ headers:
   { 'Content-Type': 'application/json',
     Authorization: 'Bearer xxxx' },
  body: 'null',
  url: 'https://api-flower-power-pot.parrot.com/user/v4/profile',
  method: 'GET' }
[Nov 06 2016 14:58:43]: Error in getInformationsCloud

There seems to be a problem with the new api response

I am getting the same error message


Struggling to run within pycharm

Black works totally fine from the command line - my issue is running it from within pycharm.  I've followed the instructions to install for pycharm in the README.
I get the following error when I run it
  File "/usr/local/bin/black", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 676, in main
    _verify_python3_env()
  File "/usr/local/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env
    'for mitigation steps.' + extra)
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment.  Consult http://click.pocoo.org/python3/for mitigation steps.

This system lists a couple of UTF-8 supporting locales that
you can pick from.  The following suitable locales where
discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8

I've tried fixed like such but I still get the same error
export LC_ALL=en_us.UTF-8
export LANG=en_us.UTF-8

If I run the following from within the shell of my virtualenv or my global python3 shell everything looks fine.
Python 3.6.5 (default, Apr 25 2018, 14:23:58)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-9<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getlocale()
('en_US', 'UTF-8')

Operating system: Mac OS High Sierra 10.13.4
Python version: 3.6.5
Black version: black 18.5b1

@vlasovskikh, do you have a suggestion how this can be fixed within PyCharm?

I've tried installing Black to my Python 3 interpreter on macOS and setting it up in PyCharm as an external tool. It went well, so I cannot reproduce it at the moment.
Make sure that the interpreter you've installed Black into has locale set properly. Check the first line of your black executable and run the following testing program with that interpreter by its full path:
environment_test.py
import locale
import os
from pprint import pprint

pprint(dict(os.environ))
pprint(locale.getlocale())

import click

@click.command()
def main():
    print("Click works well")

if __name__ == "__main__":
    main()

Then you can set up this testing program as an external tool in PyCharm in order to test your environment variables and your locale.
Did it help you with finding the source of the problem?

@ambv BTW there is a third-party black-pycharm plugin for PyCharm, but apparently it has the same issue at least on some environments, see https://github.com/pablogsal/black-pycharm/issues/2.

@kindjacket we are waiting for your report on whether you found the source of your issue.

I had the same issue, and now solved. The steps were the suspected ones:

Install python 3.6 through brew: brew update && brew install python3
sudo pip3 install black
Check black command works with No paths given. Nothing to do 😴
Create external tool in PyCharm (see below)

As an external tool it works:

With pycharm-black extension it happens as described here: https://github.com/pablogsal/black-pycharm/issues/2, it picks Java's Runtime to perform it what it seems to not work well in MacOS: https://github.com/pablogsal/black-pycharm/blob/master/src/ReformatCode.java#L53 (God's know why, even if you have generally set LANG and LC's well configured). The only workaround to make the extension work is to modify /usr/local/bin/black and set locales in Python by hand 😅 I did it and then uninstalled the extension.

@danigosa Could you try my testing script for your Python installed via brew please?

@vlasovskikh
 $ python3
Python 3.6.5 (default, May 19 2018, 06:01:45)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-9<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
 $ python3 test.py
{'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.6Af6MzDAF0/Render',
 'COLORFGBG': '15;0',
 'COLORTERM': 'truecolor',
 'DISPLAY': '/private/tmp/com.apple.launchd.bB5Y1FhpDw/org.macosforge.xquartz:0',
 'HOME': '/Users/demo',
 'ITERM_PROFILE': 'Default',
 'ITERM_SESSION_ID': 'w1t0p1:995496BD-4C52-4BDA-9A1D-0C0B01B34ED1',
 'LANG': 'en_US.UTF-8',
 'LC_ALL': '',
 'LESS': '-R',
 'LOGNAME': 'demo',
 'LSCOLORS': 'Gxfxcxdxbxegedabagacad',
 'OLDPWD': '/Users/demo/Dropbox/workspace/robbie/robbie-services/robbie-profiles-services',
 'PAGER': 'less',
 'PATH': '/Users/demo/tmp/google-cloud-sdk/bin:/Users/demo/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/opt/X11/bin',
 'PWD': '/Users/demo/Dropbox/workspace/robbie/robbie-services/robbie-profiles-services',
 'SHELL': '/bin/zsh',
 'SHLVL': '1',
 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.ihIWNbgFnF/Listeners',
 'TERM': 'xterm-256color',
 'TERM_PROGRAM': 'iTerm.app',
 'TERM_PROGRAM_VERSION': '3.1.6beta5',
 'TERM_SESSION_ID': 'w1t0p1:995496BD-4C52-4BDA-9A1D-0C0B01B34ED1',
 'TMPDIR': '/var/folders/wh/5r8gfrpx07z_rdwtsr8q3l9h0000gn/T/',
 'USER': 'demo',
 'XPC_FLAGS': '0x0',
 'XPC_SERVICE_NAME': '0',
 'ZSH': '/Users/demo/.oh-my-zsh',
 '_': '/usr/local/bin/python3',
 '__CF_USER_TEXT_ENCODING': '0x1F5:0x0:0x0',
 '__PYVENV_LAUNCHER__': '/usr/local/bin/python3'}
('en_US', 'UTF-8')
Click works well

@danigosa Have you set up this testing script as an external tool in PyCharm (as the instructions for Black recommend)? Based on your output I guess you've tried it in an external terminal or the PyCharm console or the PyCharm terminal.
This testing script acts as a substitute for Black so we can learn what's wrong with your environment when you run Black instead of it in the exactly the same manner.

I was able to fix this by explicitly setting the LANG environment variable in the black file (e.g., /usr/local/bin/black)
# -*- coding: utf-8 -*-
import re
import sys
import os

from black import main

if __name__ == '__main__':

    os.environ['LANG'] = 'en_US.utf-8'
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

@jacobzweig, this silences the exception but it doesn't fix the problem that Click is pointing out. You will not be able to properly reformat files with non-ASCII letters in the filename.

@vlasovskikh nope, I ran it directly from iTerm (with zsh). It works for me as an External Tool and in the command-line, what it does not work is the plugin for Pycharm.

I had a similar issue running black under the latest python extension for VSC (now that it supports it). And I run zsh as my shell. Once I modified my .zshrc file by adding (or uncommenting) the line export LANG=en_US.UTF-8 ... I restarted VSC and black began to work (no more click errors). So perhaps for those with PyCharm and zsh etc... maybe this will help.

@vlasovskikh impossible to reproduce in the PyCharm plugin this is all I got from running (even with your script):
BlackPycharm: Traceback (most recent call last): File "/usr/local/bin/black", line 18, in   main() File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 676, in main _verify_python3_env() File "/usr/local/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env 'for mitigation steps.' + extra) RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps.  This system lists a couple of UTF-8 supporting locales that you can pick from. The following suitable locales where discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8

+1 same problem for my Mac (used to work) but it works on my Windows 10 machine

Just tested with pycharm (2018.1.4 and macOS High sierra 10.13.5)
works perfectly in pycharm using black 18.6b2
didn't install the pycharm plugin
installed via:
(python3 already installed with brew)
Install via
$ pip3 install black
Test in terminal with:
$ black
>No paths given. Nothing to do 😴
Add to pycharm via setting-> external tool

As another data point, I'm having the same problem on macOS 10.13.15 with Visual Studio Code 1.24.1 using the new built-in formatter option for black. The output of the test script by @vlasovskikh looks normal.

@yunti thanks. I tried again using "external tool" and it works for now. 👍👍
I dug into the error messages and found the problem is from "click".
It seems nothing will be done from the click side according to the discussions in its issue tracker. 🙀💀 If Python3 is the problem, is it possible to find an alternative (for click, not python 🤗)?

The thing is, Click is right in the fact that if LC_ALL is misconfigured then Python cannot open files with paths that are outside of the misconfigured ASCII charset.
Python 3.7 fixes this by defaulting to UTF-8 anyway.
And since it's unlikely people will use Unicode names in Python paths, I will just silence this warning for the next release of Black.

Another option for PyCharm users is to use the LSP Support plugin (untested) with the Python Language Server and pyls-black plugin.

I have this problem as well. The locale test script from  @vlasovskikh raised an error "ValueError: unknown locale: UTF-8".
The fix was to add these lines to .bash_profile to explicitly define the locale:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

I noticed in my environment that LC_CTYPE was set to en_CH.UTF-8. Just including this in case the non-US locale might have caused problems.

@DylanMuir I believe the most recent version of Black already includes the workaround mentioned above and implemented by @ambv.


.get() Doesn't Return Key

Hi all,
Firstly, I really like this library. Very very nice to have something which is super simple and handles everything.
I just started using this in a production application, and noticed an issue when performing .get() requests -- the key value isn't returned properly as I'd expect.
Here's what I'm doing:
var memjs = require('memjs');

var cache = memjs.Client.create();

setInterval(function() {
  cache.get('some-key', function(err, value, key) {
    if (err) throw err;
    console.log('key:', key.toString());
    console.log('value:', value.toString());
  });
}, 60000);

What I get is a proper value (as I'd expect), but no Key value returned (eg: 'some-key'). It feels like this is a bug, as the docs suggest that key should contain the returned key name.

Thanks for pointing this out! This is a bug in the documentation. The second parameter to the callback is supposed to be the value flags. There is currently no way in memjs to set flags for a value (although there should be), so this argument will almost certainly be empty (unless a different client set the value flags).


database pathfinder installation error

SQL-запрос:
--
-- Database: pathfinder
--
-- Truncate table before insert wormhole
TRUNCATE TABLE  wormhole ;
#1146 - Table 'pathfinder.wormhole' doesn't exist

:grey_question: Bug details required
Thanks for your bug report. Unfortunately I couldn't verify/reproduce this. :disappointed:
Could you please provide us with some detailed information about this issue...

Version: Which version of Pathfinder is being used? (e.g. v1.1.2)
Server: Which server/installation is being used? (e.g. "private" or "public")
Browser: On which browser have you determined this issue? (e.g. "Chrome v52")
Frequency: How often have you seen this bug? (e.g. "This happens any time a user clicks on a System.")
Reproducible: If you found a way to reproduce this, please tell us how.

If this bug is on a "private" server/installation, please add additional environment information that could be helpful:

Web-Server: (e.g. "Nginx", "Apache")
PHP: (e.g. v7.0.8)
Database: (e.g "MySql v5.6")

Auto-generated response

pathfinder-1.1.3
Web-Server: Apache
PHP 5.6
Database mariadb 5 ,mariadb10, mysql
ubuntu 14.04
I am doing import Databases; through phpmyadmin through the installer / setup
also embed SQL Dump
one and the same error is set to an empty server apache php5.6 installed MariaDB database and MySql is always one and the same error
Database: pathfinder

Truncate table before insert wormhole

TRUNCATE TABLE wormhole;
dobovlenii wormhole at the table manually before it passes sleduyschey table and throws an error
and eve_citadel_min.sql table is imported properly
you can throw a table dump

@fackir is this update ? If not then pathfinder DB should be empty, no tables there.

the database is empty put on an empty server

On setup page is everything in green ?

yes all the basic settings in the green today porprobval once again all set to install with no problems ..... he did not understand what the Problem Thank you for your assistance will help the development of our corporation)))


Release grpc 1.31

So, the update to the build system in grpc 1.31 broke our builds. I tried my hardest to figure it out, but cmake isn't my forte.
The good news is that most of it is hosted in APK for alpine - I updated to pull those packages. One thing to note about this approach: the edge channel which has the latest & greatest versions of grpc won't keep the packages around forever. So if we want to do it this way, we're going to have to stay on top of grpc/protobuf releases, and push them out ourselves.
If you don't like using the packages, here is the APK build to get you started.

@ido-namely @vtopc this isn't done - the binaries need to be sorted in the final image, but before I finished the work, I wanted to check with you about the change in direction for how we get the binaries in the image.

@ido-namely @vtopc this isn't done - the binaries need to be sorted in the final image, but before I finished the work, I wanted to check with you about the change in direction for how we get the binaries in the image.

@abe545  I think this is the right move. it would significantly help with maintaining this repo. I appreciate you working through this.

Sorry, cmake isn't my forte too :(

https://github.com/grpckit/grpckit/blob/master/Dockerfile#L31

https://github.com/grpckit/grpckit/blob/master/Dockerfile#L31

Thanks @mhamrah! I see you guys switched to the debian base image - would probably make all this easier... The grpc documentation suggests using bazel to build, but that doesn't have packages available for alpine, so the debian image would make using their favored build system easier 🤷

Yeah I was fighting for a bit, and gave up. I like the Debian slim images.
On Wed, Aug 26, 2020, 12:56 PM Vilen Topchii<EMAIL_ADDRESS>wrote:

@vtopc approved this pull request.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/namely/docker-protoc/pull/190#pullrequestreview-475681446,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAWIGV3BBLH7TZ36B5VCWDSCU5CJANCNFSM4QFDUMYQ
.


fix: include .d.ts extension in multiple outputs

Fix for this comment https://github.com/qmhc/vite-plugin-dts/issues/136#issuecomment-1356681237
Perhaps it also fixes this issue https://github.com/qmhc/vite-plugin-dts/issues/144
Problem
With multiple entries, name equals the name of the input, which doesn't include any file extension. Therefore, the tsRe regex doesn't work.
Fix
Add .d.ts extension manually to these entries

Can confirm this happens with multiple entry files. A workaround currently is to use the beforeWriteFile hook:
dts({
  insertTypesEntry: true,
  beforeWriteFile(filePath) {
    return { filePath: filePath.endsWith('.d.ts') ? filePath : `${filePath}.d.ts` }
  },
}),


Feat/add journey outcome predictor resource

Adding a new resource Journey Outcome Predictor. This resource is linked to the Journey Outcome and references the outcomeId - needs to have a valid Outcome to be able to be created.
Swagger docs https://journey-outcome-scores.prv-use1-ai.test-pure.cloud/api-docs/#/
Have added the resource, proxy, schema and test files.
Is there a way to test this locally to see if I can create a predictor using this new code?

Thanks @charliecon.
I have been able to run and test this locally and the Outcome and Outcome Predictor were created successfully

Have made all the updates from the comments.


Episode Suggestion: The "Claim Check" Pattern

Cover this Cloud Design Pattern as part of the "Architecting for the Cloud, one deployment pattern at a time" series.
Suggestion submitted through the form at https://forms.office.com/Pages/ResponsePage.aspx?id=I8_hAe_rSkSatzbwI0Rs1nhBo3a5mW9AuJOwL1Kt0n9UQVcyOFZFNkdXU0FGNTM0Uk0zUEdPT1VFNy4u

@chrisreddington this is an interesting, worked with some of these type systems within telecom industry. I will need to get re-acquainted on this and maybe late spring?


⚠ Usage of environment variables for the API key ?

Maybe you should use an environment variable for you api_key value @public/index.html and refer to it in this file.
All you have to do are the following steps:

Create an .env file in your root directory
Create an API_KEY variable in this .env file as such: API_KEY = YOUR_API_KEY_NUMBER
Replace your api_key value inside the file index.html by: ${process.env.API_KEY}
Save both files and see the result

@doriandevtech is this project still active


debougout is not defined

Hello,
In a js file (node.js) with a function I create a new debugout object like is suggested in the documentation.
After I use grunt with my project I get the following error: debugout is not defined.
Can you help me? I am really new in javascripts

That means the debugout js file isn't getting included in your project. Check your linking. Good luck!


Angular 2.0 Tutorial!?

Awesome, I can't wait to walk through it. You guys gave me my start on Angular 1 :)

Lots of Angular 2 incoming! Awesome to hear you started with us back on ng1!