JS and TS language service die on template literal expression

Issue Type: Bug

Open VS Code, create a blank file, and set the language for the file to TypeScript.
Paste in the following code which contains the problematic template literal expression (&$filter=${string}):

export abstract class RecordStore<T> {

    constructor() {}

    async RetrieveMultiple(filter?: `&$filter=${string}` /* this template literal causes TS lang service to crash */) {
        if (this.LogicalName === "optionset") {
            debugger; // not valid
            return [];
        }

        let options = [
            `?$select=${this.QuerySelect}`,
            this.QueryExpand ? `&$expand=${this.QueryExpand}` : '',
            filter ?? "",
            this.QueryOrderBy ? `&$orderby=${this.QueryOrderBy}` : ''
        ].join('');

        return Xrm.WebApi.retrieveMultipleRecords(this.LogicalName, options).then(
            (results) => {
                let storedRecords = this.StoreDatabaseRecords(results.entities);
                return storedRecords;
            },
            (error) => {
                Controls.Messages.NotifyUser(error.message, "ERROR",
                    this.LogicalName + "_retrieveMultipleError");
                //Xrm.Utility.alertDialog(error.messages, <any>null);
                console.log(error);
                return [];
            }
        );
    }
}

Eventually VS shows error: "The TypeScript language service died 5 times right after it got started.  The service will not be restarted."

VS Code version: Code 1.53.2 (622cb03f7e070a9670c94bae1a45d78d7181fbd4, 2021-02-11T11:48:20.056Z)
OS version: Windows_NT ia32 10.0.18363
Extensions (7)

Extension
Author (truncated)
Version

Bookmarks
ale
13.0.1

vscode-eslint
dba
2.1.14

prettier-vscode
esb
5.9.2

vscode-todo-plus
fab
4.18.2

csharp
ms-
1.23.9

vs-keybindings
ms-
0.2.0

vscode-todo-highlight
way
1.0.4

(4 theme extensions excluded)

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383:30185418
vspyt653:30253241
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstry244:30256636
pythonvsdeb440:30248342
pythonvsded773:30248341
pythonvspyt600cf:30251589

Doesn't repro for me, but sounds funny. Are you doing anything special like hovering something or autocomplete? Is this the whole code required for the crash? Does this occur when all extensions are disabled? (code --disable-extensions)

Yes, it happens even with extensions disabled.  Yes, this is the only thing you need to do to cause the extension to crash, just paste in that code into a blank file that is set to TypeScript.  Code shows in the status bar that I'm on TS version 4.1.5, are you on the same version?

Sometimes I see https://github.com/microsoft/TypeScript/issues/42613, however this is not specific to the code itself
I can't get the server to crash multiple times. This is with TS 4.1.5 with all extensions disabled. @polshgiant Do you see this in every project? What about in a new workspace with just that file?

@mjbvz Hmmph.  Yes it happens in a brand new workspace with extensions disabled, just that TS file.  I can do a screen recording if that would help...where would be a good place to post it though?

Please try collecting the TS Server log from the crashed instance:

Set "typescript.tsserver.log": "verbose"

Restart VS Code and reproduce the problem

Open the TypeScript section of the output panel

At the very top, find the first line that prints to path to the semantic error log file. It should look something like:
[Info  - 19:54:59.247] <semantic> Log file: /Users/matb/Library/Application Support/Code - Insiders/logs/20200213T104930/exthost55/vscode.typescript-language-features/tsserver-log-ZT2zau/tsserver.log

That file contains the typescript logs.

Look through that log file for the first error or stack trace you see. If you can share the log, I can also take a look to see if anything stands out

⚠Warning: The TypeScript log may include information from your workspace, including file paths and source code. If you have any concerns about posting this publicly on Github, just let me know and we can arrange something else. On our side, we only use these logs to investigate issues like this

@mjbvz In the output window I see:
[Info  - 22:17:34.70] Starting TS Server
[Info  - 22:17:34.70] Using tsserver from: c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\node_modules\typescript\lib\tsserver.js
[Info  - 22:17:34.72]  Log file: c:\Users\Polshgiant\AppData\Roaming\Code\logs\20210222T072428\exthost4\vscode.typescript-language-features\tsserver-log-tIAq8O\tsserver.log
[Info  - 22:17:34.72]  Forking...
[Info  - 22:17:34.80]  Starting...
[Info  - 22:17:34.81]  Log file: c:\Users\Polshgiant\AppData\Roaming\Code\logs\20210222T072428\exthost4\vscode.typescript-language-features\tsserver-log-dNM0CC\tsserver.log
[Info  - 22:17:34.81]  Forking...
[Info  - 22:17:34.92]  Starting...
[Error  - 22:17:39.296] TSServer exited with code:<PHONE_NUMBER>
and this repeats several times
Here is one of the semantic log files, nothing obvious is jumping out at me:
Info 0    [16:17:46.460] Starting TS Server
Info 1    [16:17:46.461] Version: 4.1.5
Info 2    [16:17:46.461] Arguments: C:\Program Files (x86)\Microsoft VS Code\Code.exe c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\node_modules\typescript\lib\tsserver.js --useInferredProjectPerProjectRoot --enableTelemetry --cancellationPipeName C:\Users\POLSHG~1\AppData\Local\Temp\vscode-typescript\c8a3c6f43a90ecb9ebca\tscancellation-b9c3105646e27c09d20a.tmp* --logVerbosity verbose --logFile c:\Users\Polshgiant\AppData\Roaming\Code\logs\20210222T072428\exthost4\vscode.typescript-language-features\tsserver-log-ORkZI1\tsserver.log --globalPlugins typescript-vscode-sh-plugin --pluginProbeLocations c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript-language-features --locale en --noGetErrOnBackgroundUpdate --validateDefaultNpmLocation
Info 3    [16:17:46.486] Platform: win32 NodeVersion: 12 CaseSensitive: false
Info 4    [16:17:46.486] ServerMode: undefined syntaxOnly: false hasUnknownServerMode: undefined
Info 5    [16:17:46.488] Binding...
Info 6    [16:17:46.502] event:
{"seq":0,"type":"event","event":"typingsInstallerPid","body":{"pid":10248}}
Info 7    [16:17:46.504] request:
{"seq":0,"type":"request","command":"configure","arguments":{"hostInfo":"vscode","preferences":{"providePrefixAndSuffixTextForRename":true,"allowRenameOfImportPath":true,"includePackageJsonAutoImports":"auto"},"watchOptions":{}}}
Info 8    [16:17:46.504] Host information vscode
Info 9    [16:17:46.504] Host watch options changed to undefined, it will be take effect for next watches.
Info 10   [16:17:46.505] response:
{"seq":0,"type":"response","command":"configure","request_seq":0,"success":true}
Perf 11   [16:17:46.505] 0::configure: async elapsed time (in milliseconds) 0.5708
Info 12   [16:17:46.505] request:
{"seq":1,"type":"request","command":"compilerOptionsForInferredProjects","arguments":{"options":{"module":"commonjs","target":"es2016","jsx":"preserve","strictFunctionTypes":true,"sourceMap":true,"allowJs":true,"allowSyntheticDefaultImports":true,"allowNonTsExtensions":true,"resolveJsonModule":true}}}
Info 13   [16:17:46.506] Scheduled: ensureProjectForOpenFiles
Perf 14   [16:17:46.506] 1::compilerOptionsForInferredProjects: elapsed time (in milliseconds) 0.8706
Info 15   [16:17:46.506] response:
{"seq":0,"type":"response","command":"compilerOptionsForInferredProjects","request_seq":1,"success":true,"body":true}
Info 16   [16:17:46.506] request:
{"seq":2,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"^untitled:Untitled-1","fileContent":"export abstract class RecordStore {\r\n    \r\n    constructor() {}\r\n\r\n    async RetrieveMultiple(filter?: &$filter=${string} /* this template literal causes TS lang service to crash /) {\r\n        if (this.LogicalName === "optionset") {\r\n            debugger; // not valid\r\n            return [];\r\n        }\r\n\r\n        let options = [\r\n            ?$select=${this.QuerySelect},\r\n            this.QueryExpand ? &$expand=${this.QueryExpand} : '',\r\n            filter ?? "",\r\n            this.QueryOrderBy ? &$orderby=${this.QueryOrderBy} : ''\r\n        ].join('');\r\n        \r\n        return Xrm.WebApi.retrieveMultipleRecords(this.LogicalName, options).then(\r\n            (results) => {\r\n                let storedRecords = this.StoreDatabaseRecords(results.entities);\r\n                return storedRecords;\r\n            },\r\n            (error) => {\r\n                Controls.Messages.NotifyUser(error.message, "ERROR",\r\n                    this.LogicalName + "_retrieveMultipleError");\r\n                //Xrm.Utility.alertDialog(error.messages, null);\r\n                console.log(error);\r\n                return [];\r\n            }\r\n        );\r\n    }\r\n}","scriptKindName":"TS"}]}}
Info 17   [16:17:46.507] Search path:
Info 18   [16:17:46.507] For info: ^untitled:Untitled-1 :: No config files found.
Info 19   [16:17:46.510] Loading global plugin typescript-vscode-sh-plugin
Info 20   [16:17:46.510] Enabling plugin typescript-vscode-sh-plugin from candidate paths: c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript-language-features,c:/Program Files (x86)/Microsoft VS Code/resources/app/extensions/node_modules/typescript/lib/tsserver.js/../../..
Info 21   [16:17:46.510] Loading typescript-vscode-sh-plugin from c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript-language-features (resolved to c:/Program Files (x86)/Microsoft VS Code/resources/app/extensions/typescript-language-features/node_modules)
Info 22   [16:17:46.523] typescript-vscode-sh-plugin initialized. Intercepting getEncodedSemanticClassifications and getEncodedSyntacticClassifications.
Info 23   [16:17:46.523] Plugin validation succeded
Info 24   [16:17:46.524] Starting updateGraphWorker: Project: /dev/null/inferredProject1
Info 25   [16:17:46.539] DirectoryWatcher:: Added:: WatchInfo: c:/program files (x86)/microsoft vs code/resources/app/extensions/node_modules 1 undefined Project:  WatchType: node_modules for closed script infos in them
Info 26   [16:17:46.540] c for path c:/program files (x86)/microsoft vs code/resources/app/extensions/node_modules not found in cache...
Info 27   [16:17:46.540] Starting C:\Program Files (x86)\Microsoft VS Code\Code.exe with args:
["c:/Program Files (x86)/Microsoft VS Code/resources/app/extensions/node_modules/typescript/lib/watchGuard.js","c:/program files (x86)/microsoft vs code/resources/app/extensions/node_modules"]
Info 28   [16:17:46.667] WatchGuard for path c:/program files (x86)/microsoft vs code/resources/app/extensions/node_modules returned: OK
Info 29   [16:17:46.669] Elapsed::<PHONE_NUMBER>4926453ms DirectoryWatcher:: Added:: WatchInfo: c:/program files (x86)/microsoft vs code/resources/app/extensions/node_modules 1 undefined Project:  WatchType: node_modules for closed script infos in them
Info 30   [16:17:47.772] DirectoryWatcher:: Added:: WatchInfo: C:/Program Files (x86)/Microsoft VS Code/node_modules/@types 1 undefined Project: /dev/null/inferredProject1* WatchType: Type roots
Info 31   [16:17:47.772] watchDirectory for C:/Program Files (x86)/Microsoft VS Code/node_modules/@types uses cached drive information.
 Common extensions are ESLint & Prettier, but I tried with both and still no repro..


Cannot commit changes

Issue Summary
When I go to save a change, I get this error:
Unknown column 'id' in 'where clause'

Steps to Reproduce

Connect to database
View cell, press c to edit, modify text
Press ctrl+s to try and commit change

This is indeed a very bad problem with the way all the logic for DML statements is written, i'm working on a refactor where an id column is not needed to perform DML statements.


feat(intersection): yield initially on green light

Description
When ego vehicle entered the intersection on green traffic light collision is not detected instantly because NPC vehicles are very slow. This PR adds the feature to detect whatever vehicles in the attention area and stop for a while.
Related links
https://tier4.atlassian.net/browse/RT0-29284
launcher PR: https://github.com/autowarefoundation/autoware_launch/pull/623
Tests performed
In the attached video [00:07], "intersection" stop line is inserted even if the ego/NPCs are very slow.
https://github.com/autowarefoundation/autoware.universe/assets/28677420/cc57f702-f45b-411d-a9f2-d6cca65dba01
evaluator: https://evaluation.tier4.jp/evaluation/reports/ec7990b5-f3b7-549c-822d-22c3f72201f1?project_id=prd_jt
Notes for reviewers

Interface changes
none.
Effects on system behavior
none.
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[X] I've confirmed the contribution guidelines.
[X] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[X] The PR follows the pull request guidelines.
[X] The PR has been properly tested.
[X] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[X] There are no open discussions or they are tracked via tickets.
[X] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

It seems that the ego vehicle stops in vain if the corresponding allow signal is green.


Variadic routes take precedence over more specific routes with parameters

'/:not-found...': component1

will take precedence over:
'/section/:id': component2

which will use component1 when http://x.com/section/1
(and stangely component2 when http://x.com/section/:id)
EDIT: I just read more carefully the doc and it states that Note that Mithril checks for route matches in the order the routes are defined but isn't there no guarantees of objects elements order in a map in javascript? I also populate my routes dynamically with if and elses in an object..

The Javascript spec technically doesn't guarantee order for object iteration, but all the major js implementations behave as if it did.

@lhorie ES6 now requires it BTW, for this reason (compatibility). V8 actually ran into problems with doing otherwise, because most of the other engines did this, most notably FF and IE.


⬆ kotlinx.serialization 1.6.2

📝 Description
For aligning with Kotlin 1.9.22, we would like to upgrade the kotlinx.serialization library from 1.6.0 to 1.6.2.
🔗 Dependencies
This issue is blocked by the following ones:

[x] #409
[x] #410

✅ Checklist

[ ] Document what's new in kotlinx.serialization 1.6.1 in comment below.
[ ] Document what's new in kotlinx.serialization 1.6.2 in comment below.
[ ] Upgrade the library, update badge in README, update the dependency compatibility and the unreleased changelog.
[ ] Refactor code if needed.
[ ] Close this issue as completed and update tracking ones if present.

Another issue for upgrading to Kotlin 1.9.23 and aligning other dependencies to this new version will be created while resolving #407.


V2 Docker compose and all-in-one support

Fixes to get the docker compose and all in one deployments working for the Angular 2 version.

@richard-cox This should now fix standup-dev-env


Manual install broken

Problem
The manual install doesn't mention the projects dependencies like the core-php-client which itself has a broken manual install because its mentioned autoloader.php doesn't exist.  Therefore, the manual install cannot be completed.
Workaround:

Use composer to get a clean setup project
Rename and copy the generated folder vendor into your project's library folder or the like.
Require the autoloader.php in you project

The same problem might be mentionded in #38.


fix: encoding OData get by key

Related to https://github.com/SAP/cloud-sdk-js/issues/1398.
Closes SAP/cloud-sdk-backlog#200.

@FrankEssenberger , I rearranged the order of some sections and kept the long details into background.

@FrankEssenberger , I'm going to merge this afternoon, and you still have some time to say no;)


Add ContentType

Add ContentType in fs.info, can be used with mimetypes package
Fixes #569

Please fix the test s3fs/tests/test_s3fs.py::test_info - that this fails now is a good thing, showing that your change makes a difference.

Thank you


One lang page

What is the best way to redirect the user to another page when the page is not available in the selected language?
Ex:
/en/blog/my-article => available
/fr/blog/my-article => not available
If the user changes the lang on  /en/blog/my-article page what's the logic?
Do we need to add this condition to the select component or higher in the app?

If you do not need to next export you can use the redirects feature of next.js for this. If you need to next export you can use some logic as:
const langsNotAvailable = new Set(['fr', 'it'])
const { lang } = useTranslation()

if(langsNotAvailable.has(lang) && typeof window !== 'undefined') {
  Router.replace('/en/blog/my-article')
}

I put it outside a useEffect to don't wait to mount the component.


HC Interfaces or CH Interactions? Which one and why not be consistent?

Within the End User and User Experience Category, we have Computer-Human Interfaces and Human-Computer Interactions as groups. How are these different and why don't we use one convention (on what basis)?

We agreed that Bill and/or Barb will combine all the five low-level terms in Computer-Human Interfaces and Human-Computer Interactions into one mid-level term (group) called "Human-Computer Interaction"

Bill - I highlighted these on the "New Low Level Organized" tab in our bucket sheet.  We need to resolve the two 'online front end' terms.

merged old groups into "human-computer interaction" | aliased "online front-end" to "online front end"


Container fails to start due to can't find bundle files in tmp/bundle

When I push the application, the container installs all gems in tmp/bundle, with no errors. But the docker log file presents an error regarding to startup. The error is the container can't find the gem files, especially the puma, in tmp/bundle. The files are in the location but the container is not able to find.
Any clues?

I opened in incidente with Azure Support. They replicate the error and now the issue was addressed to Product Staff. As soon they answer the solution I'll put the answer.

This could be related to a change made in the ruby container, if you change your build to use the very latest ruby container I think it is now fixed.


Managing State Tutorial Crashes Galaxy s6

I tried running the Managing State Tutorial on my Galaxy S6 Edge and I can use the app for a few seconds and then it will either show a black screen or a red screen.  It will crash whether I use the button to increment the counter or not, but it will increment.
Here is the code I am running:
import 'package:flutter/material.dart';

void main() {
  runApp(
    new MaterialApp(
      title: 'Flutter Leads',
      routes: {
        '/': (RouteArguments args) => new Counter()
      }
    )
  );
}

class Counter extends StatefulComponent {
  _CounterState createState() => new _CounterState();
}

class _CounterState extends State<Counter> {
  int _count = 0;

  void _increment() {
    setState(() { ++_count; });
  }

  Widget build(BuildContext context) {
    return new Row([
      new RaisedButton(
        onPressed: _increment,
        child: new Text('Increment')
      ),
      new Text('Count: $_count')
    ]);
  }
}

This is the error I got:
android: I/sky     : Exception caught while building Navigator-[GlobalObjectKey _MaterialAppState(43269391)]
android: I/sky     : Bad state: No element
android: I/sky     : Stack trace:
android: I/sky     : #0      List.first (dart:core-patch/growable_array.dart:206)
android: I/sky     : #1      NavigatorState.build (packages/flutter/src/widgets/navigator.dart:119)
android: I/sky     : #2      ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1050)
android: I/sky     : #3      BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #4      StatefulComponentElement.update (packages/flutter/src/widgets/framework.dart:1149)
android: I/sky     : #5      Element.updateChild (packages/flutter/src/widgets/framework.dart:637)
android: I/sky     : #6      ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1062)
android: I/sky     : #7      BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #8      StatelessComponentElement.update (packages/flutter/src/widgets/framework.dart:1095)
android: I/sky     : #9      Element.updateChild (packages/flutter/src/widgets/framework.dart:637)
android: I/sky     : #10     ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1062)
android: I/sky     : #11     BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #12     ProxyElement.update (packages/flutter/src/widgets/framework.dart:1186)
android: I/sky     : #13     Element.updateChild (packages/flutter/src/widgets/framework.dart:637)
android: I/sky     : #14     ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1062)
android: I/sky     : #15     BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #16     ProxyElement.update (packages/flutter/src/widgets/framework.dart:1186)
android: I/sky     : #17     Element.updateChild (packages/flutter/src/widgets/framework.dart:637)
android: I/sky     : #18     ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1062)
android: I/sky     : #19     BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #20     ProxyElement.update (packages/flutter/src/widgets/framework.dart:1186)
android: I/sky     : #21     Element.updateChild (packages/flutter/src/widgets/framework.dart:637)
android: I/sky     : #22     ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1062)
android: I/sky     : #23     BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #24     ProxyElement.update (packages/flutter/src/widgets/framework.dart:1186)
android: I/sky     : #25     Element.updateChild (packages/flutter/src/widgets/framework.dart:637)
android: I/sky     : #26     ComponentElement.performRebuild (packages/flutter/src/widgets/framework.dart:1062)
android: I/sky     : #27     BuildableElement.rebuild (packages/flutter/src/widgets/framework.dart:991)
android: I/sky     : #28     WidgetFlutterBinding.buildDirtyElements.<anonymous closure> (packages/flutter/src/widgets/binding.dart:55)
android: I/sky     : #29     BuildableElement.lockState (packages/flutter/src/widgets/framework.dart:924)
android: I/sky     : #30     WidgetFlutterBinding.buildDirtyElements (packages/flutter/src/widgets/binding.dart:50)
android: I/sky     : #31     WidgetFlutterBinding.beginFrame (packages/flutter/src/widgets/binding.dart:26)
android: I/sky     : #32     FlutterBinding._handlePersistentFrameCallback (packages/flutter/src/rendering/binding.dart:176)
android: I/sky     : #33     Scheduler.invokeCallback (packages/flutter/src/animation/scheduler.dart:81)
android: I/sky     : #34     Scheduler.beginFrame (packages/flutter/src/animation/scheduler.dart:67)
android: I/sky     : #35     _beginFrame (file:///src/flutter/engine/src/out/android_Release/gen/sky/bindings/Customhooks.dart:30)

Thanks for the report.  That's very strange.  We'll need to try replicating the issue on our end.

I removed the RaisedButton and changed Text to just plain text, it seems to sit there and doesn't crash, but if you tap the screen it will do some sort of fade down animation, and then the screen is black.  I am having a hard time recreating the red screen.

A ton of code has changed here. Going to close with the hopes that it's working for you. If not, we'll reopen and take a look.
Thanks for the report! Please give it a shot again and let us know.


Adds S2S VPN functionality for CloudStack.

Hi everyone,
I added some calls to the CloudStack driver to be able to create Site-to-Site VPN connectivity. Could you please review my PR?
Thanks!

LGTM, so I merged. Thanks for the patch, keep it coming.

Thank you!


PPA for linux users

Hi,
I want to install dnscrypt-proxy from ppa to keep it always up to date.
What PPA can I use?
I found:

https://launchpad.net/~anton+/+archive/ubuntu/dnscrypt
https://launchpad.net/~xuzhen666/+archive/ubuntu/dnscrypt

Thanks for your great work

Hi,
I've been successfully using https://launchpad.net/~anton+/+archive/ubuntu/dnscrypt but both look good, up to date and pretty similar.
They both share the same limitations, though:

The resolvers list is not up to date, so you may want to update your copy from https://download.dnscrypt.org/dnscrypt-proxy/
For some reason, support for plugins is not included.

great thanks, I updated the dnscrypt-proxy.csv list.
Now the best way to update my default dns resolver is to run: dnscrypt-proxy --resolver-address=<ip>[:port]
right?
Thanks

Without the package, the way to run it is dnscrypt-proxy --resolver-name=<resolver name> (or just -R <resolver name>)
But the Ubuntu package installs it with a configuration file in /etc/default/dnscrypt-proxy, and (for more recent Ubuntu version) with a systemd service dnscrypt-proxy.service that you can also edit.
This package is neat because it can use two resolvers at the same time for redundancy.
Pick resolvers close to your location, but that also matches the features you want. For example if you rely on SSHFP records to secure your ssh keys, use resolvers that support DNSSEC.

Ok thanks.
I ran: sudo dnscrypt-proxy -R d0wn-al-ns1 but now it's stuck on [NOTICE] Proxying from <IP_ADDRESS>:53 to <IP_ADDRESS>:54

Check that it works well, and then add -d to the command the keep it running as a background process

Ok thanks, anyway I have created a script to test the best DNS in terms of speed.
Do you think it is ok to use dig google.com @$ADDRESS -p $PORT to test DNS response time and find the best?
Thanks


Add optimize module

The Optimize Module performs series of optimizations on llvm module based on the sub-target (GFX9). E.g llvm::CodeGenOptLevel::Aggressive
The code is inspired by the code section in
https://github.com/openxla/iree/blob/main/compiler/plugins/target/CUDA/CUDATarget.cpp and
https://github.com/ROCmSoftwarePlatform/triton/blob/triton-mlir/lib/Target/HSACO/HSACOTranslation.cpp

It would be good if we could find some way to test this. Also can you add a brief description to the PR describing what you're trying to accomplish?

I am checking again with some test cases that might be used. Also added a short description.

It would be good if we could find some way to test this.

@qedawkins Please see below for this PR the performance differences before and after.
Before optimize module
2023-11-27T18:28:29-08:00
Running ../iree-build/tools/iree-benchmark-module
Run on (64 X 7006.64 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x32)
L1 Instruction 32 KiB (x32)
L2 Unified 512 KiB (x32)
L3 Unified 32768 KiB (x4)
Load Average: 0.54, 0.51, 0.63
WARNING CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
WARNING Library was built as DEBUG. Timings may be affected.
Benchmark                                 Time             CPU   Iterations UserCounters...
BM_matmul/process_time/real_time      48997 ms        48945 ms            1 items_per_second=0.0204094/s
After optimize module
2023-11-27T18:33:30-08:00
Running ../iree-build/tools/iree-benchmark-module
Run on (64 X 7006.64 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x32)
L1 Instruction 32 KiB (x32)
L2 Unified 512 KiB (x32)
L3 Unified 32768 KiB (x4)
Load Average: 3.26, 3.42, 1.85
WARNING CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
WARNING Library was built as DEBUG. Timings may be affected.
Benchmark                                 Time             CPU   Iterations UserCounters...
BM_matmul/process_time/real_time      45699 ms        45668 ms            1 items_per_second=0.0218825/s

Nice, that's good justification; @nirvedhmeshram seems to have more context on the llvm side, but is there any kind of reasonable e2e test worth adding here? Otherwise it would be good to log these performance numbers in the commit message as a record.

Short of adding a benchmark CI there isn't much that we can do in terms of e2e test that measures performance.
Also for the LLVM path itself, I don't know if we can add a lit test. This path is tested on all rocm tests, so it is tested, but can't really "test" for benchmark results afaik

Short of adding a benchmark CI there isn't much that we can do in terms of e2e test that measures performance. Also for the LLVM path itself, I don't know if we can add a lit test. This path is tested on all rocm tests, so it is tested, but can't really "test" for benchmark results afaik

Ok sorry for over-indexing on that then. I'm not too familiar with LLVM backend details so I will let others approve.

We can at least add compilation correctness test coverage pretty easily. Execution correctness and performance test coverage will be more tricky.

We can at least add compilation correctness test coverage pretty easily. Execution correctness and performance test coverage will be more tricky.

I think this path is effectively being tested by all rocm e2e tests...

We can at least add compilation correctness test coverage pretty easily. Execution correctness and performance test coverage will be more tricky.

I think this path is effectively being tested by all rocm e2e tests...

Can you point to those tests? I don't think they exist right now, at least not here in IREE
I saw this thread this morning and planned on adding some today.
I see these ROCm tests right now:
iree/compiler/Dialect/HAL/Target/ROCM/test/smoketest.mlir.test
iree/compiler/Codegen/LLVMGPU/test/reduction_pipeline_rocm.mlir.test
iree/compiler/Codegen/LLVMGPU/test/reduction_pipeline_transform_rocm.mlir.test
experimental/regression_suite/tests/pregenerated/test_llama2.py::test_step_rdna3_rocm_stripped

@MaheshRavishankar @ScottTodd @nirvedhmeshram, please let me know when can we merge this PR.

We can at least add compilation correctness test coverage pretty easily. Execution correctness and performance test coverage will be more tricky.

I think this path is effectively being tested by all rocm e2e tests...

Can you point to those tests? I don't think they exist right now, at least not here in IREE I saw this thread this morning and planned on adding some today.
I see these ROCm tests right now:
iree/compiler/Dialect/HAL/Target/ROCM/test/smoketest.mlir.test
iree/compiler/Codegen/LLVMGPU/test/reduction_pipeline_rocm.mlir.test
iree/compiler/Codegen/LLVMGPU/test/reduction_pipeline_transform_rocm.mlir.test
experimental/regression_suite/tests/pregenerated/test_llama2.py::test_step_rdna3_rocm_stripped

Yeah I dont think there are any e2e tests for rocm. Would be great to add those.

Looks good to me, if the cloning can be addressed as a follow up.

Sure, I think it is mentioned in this issue https://github.com/openxla/iree/issues/15698


[TRIVIAL] Clarify online delete data error message.

This was inspired by the conversation on https://github.com/ripple/rippled/pull/3015
Namely, the existing error message is unclear that rippled doesn't just automatically know how to move its data around when the config changes. This error message is more clear that rippled doesn't know how to work with data in a corrupted state.

Jenkins Build Summary
Built from this commit
Built at 20190801 - 08:07:28
Test Results

Build Type
Log
Result
Status

msvc.Debug
logfile
1178 cases, 0 failed, t: 569s
PASS :white_check_mark:

gcc.Release -Dassert=ON, MANUAL_TESTS=true
logfile
915 cases, 0 failed, t: 4m59s
PASS :white_check_mark:

msvc.Debug, NINJA_BUILD=true
logfile
1178 cases, 0 failed, t: 605s
PASS :white_check_mark:

gcc.Debug -Dcoverage=ON, TARGET=coverage_report, SKIP_TESTS=true
logfile
1178 cases, 0 failed, t: 16m47s
PASS :white_check_mark:

docs, TARGET=docs
logfile
1 cases, 0 failed, t: 0m1s
PASS :white_check_mark:

clang.Debug
logfile
1178 cases, 0 failed, t: 2m57s
PASS :white_check_mark:

msvc.Debug -Dunity=OFF
logfile
1177 cases, 0 failed, t: 1094s
PASS :white_check_mark:

msvc.Release
logfile
1178 cases, 0 failed, t: 403s
PASS :white_check_mark:

clang.Debug -Dunity=OFF
logfile
1177 cases, 0 failed, t: 11m13s
PASS :white_check_mark:

gcc.Debug
logfile
1178 cases, 0 failed, t: 2m57s
PASS :white_check_mark:

gcc.Debug -Dunity=OFF
logfile
1177 cases, 0 failed, t: 11m23s
PASS :white_check_mark:

clang.Release -Dassert=ON
logfile
1178 cases, 0 failed, t: 5m7s
PASS :white_check_mark:

gcc.Release -Dassert=ON
logfile
1178 cases, 0 failed, t: 4m59s
PASS :white_check_mark:

gcc.Debug -Dstatic=OFF
logfile
1178 cases, 0 failed, t: 2m58s
PASS :white_check_mark:

gcc.Debug -Dstatic=OFF -DBUILD_SHARED_LIBS=ON
logfile
1178 cases, 0 failed, t: 2m56s
PASS :white_check_mark:

gcc.Debug, NINJA_BUILD=true
logfile
1178 cases, 0 failed, t: 2m51s
PASS :white_check_mark:

clang.Debug -Dunity=OFF -Dsan=address, PARALLEL_TESTS=false, DEBUGGER=false
logfile
1177 cases, 0 failed, t: 11m15s
PASS :white_check_mark:

clang.Debug -Dunity=OFF -Dsan=undefined, PARALLEL_TESTS=false
logfile
1177 cases, 0 failed, t: 12m39s
PASS :white_check_mark:

rebased

Codecov Report

Merging #3016 into develop will decrease coverage by <.01%.
The diff coverage is 0%.

@@             Coverage Diff             @@
##           develop    #3016      +/-   ##
===========================================
- Coverage    68.77%   68.76%   -0.01%
===========================================
  Files          686      686
  Lines        55751    55749       -2
===========================================
- Hits         38340    38337       -3
- Misses       17411    17412       +1

Impacted Files
Coverage Δ

src/ripple/app/misc/SHAMapStoreImp.cpp
75.85% <0%> (+0.42%)
:arrow_up:

src/ripple/ledger/impl/ApplyView.cpp
80% <0%> (-2.61%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5834fbb...53ec9e3. Read the comment docs.


Board Review: Azure IoT Models Repository Client (Python)

Contacts and Timeline

Responsible service team: Azure IoT Portal & UX
Main contacts:

Developer<EMAIL_ADDRESS>Product Manager<EMAIL_ADDRESS>Dev Lead<EMAIL_ADDRESS>

Expected code complete date: March 17 2020
Expected release date: (Unknown - This is a preview version of a package)

About the Service

Link to documentation introducing/describing the service: N/A
Link to the service REST APIs: N/A
Link to GitHub issue for previous review sessions, if applicable:  N/A

About the client library

Name of the client library: Azure Iot Models Repository (azure-iot-modelsrepository)
Languages for this review: Python

Artifacts required (per language)
Python

APIView Link: https://apiview.dev/Assemblies/Review/d043acebf7b7407fb4533d9814d1e334
Link to Champion Scenarios/Quickstart samples: Link to samples
PR: https://github.com/Azure/azure-sdk-for-python/pull/17180

reviewed with Java library 3/29


make bs4 book theme available

On Basecamp we were talking about using the bookdown::bs4_book() theme - it looks like this is currently only available in the development version of bookdown, so we'd need to install bookdown from github until v0.22 is released

We can add this as a possibility in the Dockerfile and document how to switch that in getting_started.md. I generally like to avoid development versions until they are officially determined to be stable. Do we know when v0.22 should be released approximately?

Yeah, I understand wanting to be careful about using dev versions.
From a quick look around the bookdown repo, I can't tell when it'll be released.

Last release was October 2020, so it's been kind-of a long gap (previous releases were coming out every ~couple of months)
Milestone for v0.22 says April 30th but I've no idea how meaningful that is, and they don't seem to really be using Milestones to track issues.

v0.22 is out!

The docker image is using bookdown 0.24 so bookdown::bs4_book() is available.


Two-phase conversion to avoid requests deadlocks. Closes #4

Hmmm, I see what you are doing, but I think the better way would probably be to add another override to premailer, so that it ignores base_url for stylesheets and just loads them from the filesystem with my current override

I will merge this, and refactor later :)

Ok. Let me know if you need any help.


Update archive locales and styles

Follow up to https://github.com/typst/hayagriva/pull/135
This PR aims to update the archive locales and styles to the latest upstream of https://github.com/citation-style-language/locales, which has the "Online" translation for pt-PT fixed as discussed in the mentioned PR.

Thank you for your upstream contribution! Since there is breakage in the APA test through the update, I instead updated just the locales in 22f4439379440e3200d024c0d7f8a9d5e48157df.


Proposal to move most attributes into relations

This is a discussion ticket that's an offshoot of #276:

When a road or a building or a lake changes its name but geometry doesn't change, can we handle that change over time via relations?

Example made by Natfoot:
https://www.openhistoricalmap.org/edit#map=16/47.8679/-122.2844:

This is a case where the geometry itself has no attributes, but it is part of 2 relations that do.
Here's one:

And the other:

And hey, wait, this is already working:
1940

1939

Is that special to lakes and water or would that work on other examples?
Either way, is it a viable option from a performance and infrastructure perspective to encourage editors to offload dynamic attributes onto relations?

in my view this should be the preferred approach going forward

Should be listed as one of the options at https://wiki.openstreetmap.org/wiki/Open_Historical_Map#Tagging?
If we promote this style of mapping I'd like to see it extended into a more generic relation which supports nodes, linear ways and closed ways.

Agreed that we need to write this up as a recommended approach, but first we want to make sure it will work with our infrastructure to encourage this kind of editing -- and also that it works for people doing the editing!
That wiki page mentioned above https://github.com/OpenHistoricalMap/issues/issues/284#issuecomment-914634033 links through to https://wiki.openstreetmap.org/wiki/Comparison_of_life_cycle_concepts. Which I hadn't looked at in a few months. It has some useful examples, especially under the proposal for dates-in-keys. Interested to think about how many relations would be required to represent this OSM example:
 highway = pedestrian
 highway:1974- = pedestrian
 highway:1932-1974 = primary
 maxspeed = 30
 maxspeed:1957-1992 = 50
 name:1871-1933 = Kaiserstraße
 name:1933-1945 = []-Straße
 name:1945-1953 = []straße
 name:1953-1990 = Ernst-Thälmann-Straße
 name:1990- = Konrad-Adenauer-Straße
 ref:1932-1960 = R 96
 ref:1960-1990 = F 96
 ref:1990- = B 96
 surface = paved
 surface:1890-1927 = cobblestone
 surface:1927- = paved

There are 13 unique combinations of start/end date tied to various attributes:

1871-1933
1890-1927
1927-
1932-1960
1932-1974
1933-1945
1945-1953
1953-1990
1957-1992
1960-1990
1974-
1990-
1992-

That's a lot of relations to make! But it is also really hard to imagine a client-side date-parser that could traverse all those keys and make sense of them out of vector tiles. Offloading that onto relations allows the timeslider and map style to remain simple enough to run client-side.
But it also suggests that the number of records in the database could grow a lot, and that there's going to be a lot of either duplication of tags or some TBD merging magic when, say, the surface of a road changes out of sync with its name, which are both out of sync with another change. It seems then like each unique combination of attributes might need its own relation.
Interestingly (to me anyway!) if I just parse out all the dates in those keys, then sort them and make non-overlapping ranges, I get a shorter and more comprehensible list:

1871-1890
1927-1932
1932-1933
1933-1945
1945-1953
1953-1957
1960-1974
1974-1990
1990-1992
1992-

Each relation would need to include the full set of tags that were true at that point, even if only some of them changed, but also if you want to know what the street is like in Year X, you just need the relation valid at that point, you don't have to hunt through an unknown number of keys to see which, if any might include years and if those years are in the period you're looking for.

i can see arguments either way on how date ranges and keys are distributed across relations. i do think that it's easier for an implementer to traverse a relation graph of some sort than it is to parse out the key-with-date format, and always remember that there are and will be data consumers that aren't the vector tile renderer. i'm thinking about animating using ephemeral data myself.

Further thinking on this: @rwelty1889 I hear you about other consumers than the vector tiles. And yet I'm gonna keep talking about vector tiles and the timeslider, since that's the bit I know best and also that we need to have the main map on OHM.org. But I'm glad you're here to keep us thinking about other consumers!
If we pursued this sort of tagging approach, where most attributes are in relations, I wonder if either (a) we would replace node/linear way/closed way vector tile layers with relations or (b) we'd need to combine or duplicate layers to pull in both. (a) might be an ideal scenario if we were starting from nothing, but (b) is more likely to be the need since we do have a lot of tagged nodes and ways.
Looking at the example on staging here: https://vtiles-staging.openhistoricalmap.org/#15.11/47.754747/-121.761209, we have 4 instances of the one segment (3 relations and 1 way):

But when editing, we have one instance of the segement that is part of three relations:
https://staging.openhistoricalmap.org/way/198316714#map=17/47.75896/-121.76474&layers=OD&date=1700&daterange=1700,2021

This is what we want based on the proposal in this issue.
One potential infrastructure issue: In the vector tiles, we end up with 4 copies of the geometry (one for each relation and one for the way). That's fine for a small segment like this, but would that scale to more complex and more numerous items? Do we quickly start hitting the limits of what is contained in a single vector tile?

Agreed that we need to write this up as a recommended approach, but first we want to make sure it will work with our infrastructure to encourage this kind of editing -- and also that it works for people doing the editing!
That wiki page mentioned above #284 (comment) links through to https://wiki.openstreetmap.org/wiki/Comparison_of_life_cycle_concepts. Which I hadn't looked at in a few months. It has some useful examples, especially under the proposal for dates-in-keys. Interested to think about how many relations would be required to represent this OSM example:
 highway = pedestrian
 highway:1974- = pedestrian
 highway:1932-1974 = primary
 maxspeed = 30
 maxspeed:1957-1992 = 50
 name:1871-1933 = Kaiserstraße
 name:1933-1945 = []-Straße
 name:1945-1953 = []straße
 name:1953-1990 = Ernst-Thälmann-Straße
 name:1990- = Konrad-Adenauer-Straße
 ref:1932-1960 = R 96
 ref:1960-1990 = F 96
 ref:1990- = B 96
 surface = paved
 surface:1890-1927 = cobblestone
 surface:1927- = paved

There are 13 unique combinations of start/end date tied to various attributes:

1871-1933
1890-1927
1927-
1932-1960
1932-1974
1933-1945
1945-1953
1953-1990
1957-1992
1960-1990
1974-
1990-
1992-

That's a lot of relations to make! But it is also really hard to imagine a client-side date-parser that could traverse all those keys and make sense of them out of vector tiles. Offloading that onto relations allows the timeslider and map style to remain simple enough to run client-side.
But it also suggests that the number of records in the database could grow a lot, and that there's going to be a lot of either duplication of tags or some TBD merging magic when, say, the surface of a road changes out of sync with its name, which are both out of sync with another change. It seems then like each unique combination of attributes might need its own relation.
Interestingly (to me anyway!) if I just parse out all the dates in those keys, then sort them and make non-overlapping ranges, I get a shorter and more comprehensible list:

1871-1890
1927-1932
1932-1933
1933-1945
1945-1953
1953-1957
1960-1974
1974-1990
1990-1992
1992-

Each relation would need to include the full set of tags that were true at that point, even if only some of them changed, but also if you want to know what the street is like in Year X, you just need the relation valid at that point, you don't have to hunt through an unknown number of keys to see which, if any might include years and if those years are in the period you're looking for.

Using a relation every time a tag is different is the best way to make verification and modification impossible and discourage potential contributors (in addition to having a way of doing things too different from OSM), using tag:DATE-DATE=value is so far the simpliest way, of course it requires some work on the renderer but it's way more simple for contributors.

two counterpoints to Dlwig's comment.

serious consideration needs to be given to the impact that dates and times embedded in keyspace will have on the speed and complexity of overpass queries, as that is one of the ways that things are going to be done, in particular, it will likely be much easier to do a date match against a start_date or end_date tag than a date match against the right side of a key

the current, dead OSM proposal for dates in key space has the problem of not supporting time format at all (and time data is coming to OHM, i've entered some already), and it is incompatible with dates either as specified on the start_date page or in EDTF/ISO 8601. so that would need to be worked through.

i think relations need much better editor support, and how we use them needs careful thought. i am currently doing the lifecycle laboratory stuff with the Erie Canal, Hudson River, and the Harmony Mill complex in Cohoes NY in an effort to see how relation based models work in practice. a combination of better relation support in ID and JOSM (lifecycle plugins) and automagic relation management tools would likely make a huge difference. i've already decided the OSM model of long linear relations to group things together breaks down badly with complex life cycles with current tools and support.

First of all, forgive me for this very late response.
Yes, using the "TAG: DATE-DATE = VALEUR" format cannot work well with overpass queries, we can make the same remark on almost all the tools which read a database with a those tags to display a map (Leaflet, QGIS ...).
Now, I think overusing relationships is just a way of putting off the problem. The more we move away from OpenStreetMap practices, the more we frighten potential contributors, not to mention that creating a relationship for each change is complex to do and almost impossible to modify and verify.
And even admitting that this would be the simplest solution, it will only postpone the inevitable ... What will happen when the query has to analyze thousands of relations to display the map, even with only START_DATE and END_DATE tags ? I doubt that the request will work correctly, and we will end up at the starting point.
Couldn't we consider using raster tiles like Openstreetmap does or vector tiles?

On 1/11/22 8:36 AM, Arflhn wrote:

First of all, forgive me for this very late response.
Yes, using the "TAG: DATE-DATE = VALEUR" format cannot work well with
overpass queries, we can make the same remark on almost all the tools
which read a database with a those tags to display a map (Leaflet, QGIS
...).
Now, I think overusing relationships is just a way of putting off the
problem. The more we move away from OpenStreetMap practices, the more we
frighten potential contributors, not to mention that creating a
relationship for each change is complex to do and almost impossible to
modify and verify.
And even admitting that this would be the simplest solution, it will
only postpone the inevitable ... What will happen when the query has to
analyze thousands of relations to display the map, even with only
START_DATE and END_DATE tags ? I doubt that the request will work
correctly, and we will end up at the starting point.

most anyone who understands DB and Query behavior is pretty cool to
embedding dates in the key side of the kv pair. this should only be a
controversial point for folks who all about data entry and not paying
attention to data consumer issues.
but step back for a moment. i totally get the sentiment about wanting
to stay with OSM practices as best as possible, but i have been thinking
about lifecycle and thinking about the relationship between OHM and OSM
quite a lot for months now (i have a proposal for a talk at SOTM US in
april on the subject) - and OSM doesn't have much to say on the topic
that's useful for OHM because lifecycle isn't a think that OSM has to
do much with.
if we use the relationship model, then what data consumers will do is a
query that returns too much (they have to) and then do a graph
traversal. there are worse things. from a data entry point of view, i
think we need better tools for relationships. i suspect that many agree
that current relationship tools in OSM editors are on the primitive
side. we can do better.
neither approach is perfect. i think relations are better. but i also
think that alternative proposals are welcome. feel free to make one.

Couldn't we consider using raster tiles like Openstreetmap does or
vector tiles?

not sure i follow this. the current OHM rendering engine uses vector
tiles. raster tiles are problematic with the time slider.
richard
@.***
Averill Park Networking - GIS & IT Consulting
OpenStreetMap - PostgreSQL - Linux
Java - Web Applications - Search

Would it be possible to use opening_hours like syntax? For example name:conditional=Kaiserstraße 1871÷1933;[]-Straße   1933÷1945; "[]straße" 1945÷1953;Ernst-Thälmann-Straße 1953÷1990;Konrad-Adenauer-Straße 1990 (note, the division symbol can be different and :conditional could be another name. It's just an example to show how it can be tagged).

We had a really good discussion around this topic at SOTM US. To summarize the problem we are trying to solve again: when the geometry of a feature does not change, but some attributes do, for eg. the name, it seems a bit terrible to have to re-draw the same feature with a different start_date and end_date with changed attributes. So, we need a good solution to this, and ideally a solution that works for:

people editing data, using JOSM / iD
the renderer which is creating the vector tiles and the styles to display the vector data
data consumers, who want to use Overpass or other tools to get the data in a reasonable format to process

It seems like the proposal to encode time data in the key part of the data maybe good for people editing the data, but it seems like a real nightmare for both the renderer and for any sort of data consumers. Having data encoded in the key side of a key value pair just seems like it's going to be very hard to integrate with the time slider as well as other query mechanisms. I would strongly pushback against such a proposal.
Encoding the data in Relations feels like a bit of a hack, but I also think this is the best and most practical solution we have. Relations in OSM work well for this kind of thing, and for most cases, the vector tile pipeline already supports this - so, to give a simple example: You could have a way with a geometry - and then multiple relations that have that way as part of the relation, with different tags for start_date, end_date and name for eg. to represent changes of name of a way. It seems like this should be handled well by most existing OSM tools like Overpass - we should really avoid re-inventing things if we can.
I think the strongest argument AGAINST encoding this data in Relations is that this can get awkward for editors, and hard for people to do who are not comfortable with fairly advanced OSM editing concepts (tbh, I'd probably have no idea how to do this properly in iD or JOSM) - but I feel way more comfortable with the idea of writing editor plugins and better UI to handle this in the editors, rather than making fundamental changes to the database / schema, etc.
I think the other consideration might be that we start packing too much data into the vector tiles, since this will create a new feature in the vector tiles for each of these Relations. Again, I feel like this is the correct place to handle this complexity rather than attempting crazy things in the database, or fundamentally altering the OSM data model - this maybe a problem down the line, but I also think we'd have a lot of options to alleviate this problem should it arise.
For me, next actions here would be something like:

Garner consensus on this approach - edit the wiki with examples to make it crystal clear how things should be created and tagged. Test examples heavily with the current renderer. If something does not work, make tickets and we should be able to work on fixing this. @rwelty1889 would you be able to help with this? @geohacker I'd lean on you heavily here to advise / veto if you think it's a terrible idea, but from my understanding, doing this in relations should mostly "just work", with perhaps just minor changes to the SQL queries for the vector tiles.
Outline a vision for what editor plugins to support this could be like, work out feasibility, and scope out building these plugins / modification to editors. This is probably a bit of work, but I think this would be the correct place to handle this. In my ideal world, with editor modifications, users should not even know or need to care about the details of the data structure / how relations are mapped to features in the backend -- they should just be able to annotate a geometry with separate start and end dates with tags for each, and the editor should handle creating the required relations, etc. This is definitely a bit pie-in-the-sky and I have limited experience with writing plugins for JOSM or modifying iD, but it feels like this should be a problem we can start working on incrementally? cc @Rub21 to advise on the feasibility of doing something like this in JOSM.

Thanks again to everyone for thinking through this and especially @rwelty1889 for the extremely lucid and well thought out presentations at SOTM US - it does seem like OHM is growing at a pace where the naive approach of just creating a new feature for every "change" in the lifecycle of a feature is clearly not the best data model. The approach of handling this with Relations, from a data structure and rendering point of view seems the best to me. It's totally understandable that this is not the best for current editors, and I feel like we should move discussion to how we improve editors to support this better - that feels to me like a much better place to solve the problem rather than trying to fundamentally change the backend data model.
Hope this all makes sense and we are able to move forward with this.
cc @rwelty1889 @jeffreyameyer @geohacker @Rub21 @danrademacher

Thank you all for this discussion! @batpad's comment above sums up the approach and it sounds to me like the right way to handle and feels like one of the appropriate use cases for relations. Just that here the relationship is differences of a feature over time. The tile server already supports this because we import all relations regardless of the tags into the database. This was a deliberate decision early on to ensure we can somewhat be resilient to tagging approach changes but in this case is a coincidence that the map renders as expected. I'd not count on this coincidence, so we should:

Get a few test cases in the OHM test bed
See the impact of these new tests cases on the tiles and style. Particularly what is breaking
Expand our vector tile layer definition to query relation and relation_members for all possible scenarios of attribute changes and expose this into the tiles

Adding a pointer to #488 for some additional background.


Feature: Add config properties for private-network mode (e.g. in case of bastion or VPN setup).

Hi,
We run Kubernetes inside private subnets. The cluster YAML is configured with mapPublicIPs: false to completely seal off the cluster from external access (== more secure). Outbound Internet access is done through a NAT, like documented. Admin (SSH mainly) is done with help of a VPN server sitting in the same VPC as the Kubernetes subnets.
To allow a service under development which is running on my local machine to talk to services on the development cluster, I had to add a rule to the worker SG and allow inbound traffic from the VPC CIDR. This is a post kube-aws manual step, so I thought maybe it's a good idea to put this in the cluster.yaml config.
Secondly, when mapPublicIPs: false is configured, the controller LoadBalancer is provisioned as internal. This makes it hard to run kubectl commands from outside the VPN, for example to do CI/CD. I wonder if there could be a config option to switch this to public.
So 2 options are requested:

[ ] vpcAccessToWorkerNodes=[true/false] => Add a inbound rule to the worker SG.
[ ] controllerLoadBalancer=[public/private] => switches the LB mode

Hi @PepijnK if you create an SG when you set up your VPC, e.g. to allow inward from VPC CIDR, you can automatically attach that SG to workers in the cluster.yaml using workerSecurityGroupIds. That is what I use to achieve what you describe.
# Existing "glue" security groups attached to worker nodes which are typically used to allow access from worker nodes to services running on an existing infrastructure
workerSecurityGroupIds:
  - sg-12345678

For the second issue, I agree it would be nice to choose whether the API is public, separate from whether the worker nodes are public.

I wonder when kube-aws turned into AWS management tool? IMHO it should focus on a kube and not on a cloud infrastructure.  Almost non of the latest PRs are about kube and about cloud management which people try to get for free from a tool which just happen to deal a bit with subnets and security groups.
IMHO kube-aws should support just 2 modes:

basic "demo-only" mode where subnets, routing tables etc are created to play around
normal mode, where ASGs are created in existing subnets.

If there is a demand to drive cloud infrastructure from some high-level yaml file, then it can be done  in a spin-off project, not in kube-aws itself. but with amount of variance that file will quickly grow into stack-template.yaml which can be directly submitted to CF :)

@redbaron I don't agree with that in many ways:

Kube-aws is a bridging tool for deploying K8s on AWS, so off course it should have parameters that makes it possible to customize your cluster.
You should regard the YAML file also as infrastructure in code. Is it the source of truth? Or the rendered CF-stack? I guess the YAML file... This means some obvious post-install tweaking in K8s management infra should be avoided and ideally made configurable in the YAML file. Rendering a CF stack and use that for tweaking, would make kube-aws an init-only tool which is IMO not the idea behind it. Editing the YAML file is also by far easier then editing the rendered CF-stack file.
Regarding modes. A predefined set of properties would be the worst solution, as it restricts very much the ability to customize the K8s infra to your needs. These modes are already there in fact: In case of DEMO use-case, simply keep the properties to their sese-making defaults.

I'm just referring to properties that are now arbitrarily chosen anyway. Making them configurable would make me happy already.

OK, lets discuss that.

Kube-aws is  bridging tool for deploying K8s on AWS, so of course it should have parameters that makes it possible to customize your cluster.

indeed kube-aws is deploying k8s on AWS, but it is not a AWS cloud management tool. It is unfortunately quicky becoming one, but from my POV this is a wrong course of development.  kube-aws should deliver quality kube setup: good RBAC defaults, common base layer bits like log collection support, prometheus monitoring, flexible overlay network backends, network and etcd multi tenancy, PKI infrastructure integration and many more.
What happens instead is that people assume that if kube-aws provisions some subnets, then it should be capable to manage their whole AWS infrastructure. No it is not.  It is just a matter of time , people will be asking for VPN gateway management, direct connect, cloudwatch alerts setup etc. Amount of different setups is so vast, that Amazon created CF to be able to describe it.  kube-aws is not a CF replacement, and shouldn't try to accommodate every possible customization into   cluster.yaml format,trying to do that inevitably will make cluster.yaml look like CF template in YAML format.
As a demo-only kube-aws should be able to spin up cluster in a new VPC with some basic ELB + ASG setup.
Normal mode of operation should be "run into existing subnets", where AWS infrastructure is created by other means and kube-aws concerned is only about creating a well behaving cluster on top of it.
Most of recent PRs and issues raised have nothing to do with k8s operation, but with AWS infrastructure it is running on. It clearly indicates that there is a desire to have a simple tool, where in a top level cluster.yaml-like file whole infrastructure is described. That's fine, but it should be moved to another sister project whose goal will be to manage AWS infrastructure in a simplified way.
I am not against of flexibility, I am against losing focus of a project.

IMHO you are a but too strict about the scope. In fact, if no customization
is allowed kube-aws would be a bit useless for me. It renders a private
ELB, which I can't make public afterwards. So I need to hack into the CF
template. Following your argumentation why bother about a tool after all?
Simply commit some demo CF templates to GitHub could prove that k8s runs on
AWS as well.
And why not could aws-kube spin up a production grade cluster? Is aws-kube
really demoted to setup a demo cluster only? I agree this tool should not
encapsulate other AWS service that has nothing or little to do with K8s,
but the stuff it creates should be configurable. That is what my CR is
about at last which somehow triggered this meta discussion ;)
Op 26 jan. 2017 14:24 schreef "Maxim Ivanov"<EMAIL_ADDRESS>
OK, lets discuss that.
Kube-aws is bridging tool for deploying K8s on AWS, so of course it should
have parameters that makes it possible to customize your cluster.
indeed kube-aws is deploying k8s on AWS, but it is not a AWS cloud
management tool. It is unfortunately quicky becoming one, but from my POV
this is a wrong course of development. kube-aws should deliver quality kube
setup: good RBAC defaults, common base layer bits like log collection
support, prometheus monitoring, flexible overlay network backends, network
and etcd multi tenancy, PKI infrastructure integration and many more.
What happens instead is that people assume that if kube-aws provisions
some subnets, then it should be capable to manage their whole AWS
infrastructure. No it is not. It is just a matter of time , people will be
asking for VPN gateway management, direct connect, cloudwatch alerts setup
etc. Amount of different setups is so vast, that Amazon created CF to be
able to describe it. kube-aws is not a CF replacement, and shouldn't try to
accommodate every possible customization into cluster.yaml format,trying
to do that inevitably will make cluster.yaml look like CF template in
YAML format.
As a demo-only kube-aws should be able to spin up cluster in a new VPC
with some basic ELB + ASG setup.
Normal mode of operation should be "run into existing subnets", where AWS
infrastructure is created by other means and kube-aws concerned is only
about creating a well behaving cluster on top of it.
Most of recent PRs and issues raised have nothing to do with k8s
operation, but with AWS infrastructure it is running on. It clearly
indicates that there is a desire to have a simple tool, where in a top
level cluster.yaml-like file whole infrastructure is described. That's
fine, but it should be moved to another sister project whose goal will be
to manage AWS infrastructure in a simplified way.
I am not against of flexibility, I am against losing focus of a project.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/coreos/kube-aws/issues/277#issuecomment-275387153,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADLJESmkKVzgqWqAMOQN1hPE3KwOE-sqks5rWJ6FgaJpZM4LrCS_
.

This is a difficult subject. The aim is to provide a user-customizable framework to deploy production ready platform, without overwhelming new users with thousands of knobs to fiddle with. Maintenance/flexibility suffers from having too many config options too.

So I need to hack into the CF template.

That's exactly why the steps "generate template" and "deploy cluster" are separate - to allow you to do this.

indeed kube-aws is deploying k8s on AWS, but it is not a AWS cloud management tool.

Agreed here as well.

IMHO you are a but too strict about the scope.

This is at times necessary. Scope and focus is important! We don't want to burn out the one maintainer that somehow keeps finding time for this project. The aim is to make the rendered outputs as customizable as possible and as far as i can tell, your specific usecase can currently be solved with workerSecurityGroupIds and a little tweak to the CF template.
Discussing code makes things a lot easier though ;-)

OK, clear. Not want to put down the maintainer. The tool is doing a great job, that is why I like to use it. Good work has been done here. I need to consider the YAML file isn't supposed to be the single source of truth and is disposable. The rendered CF template is leading.

Hi @PepijnK I just commit the patches to the generated code. At any given time I may or may not need patches due to bugs I'm trying to fix, deployment choices that I can't make in cluster.yaml, and totally local-specific customizations.
kube-aws render stack
[[ -r stack-template.patch ]] && patch stack-template.json stack-template.patch
[[ -r cloud-config-controller.patch ]] && patch userdata/cloud-config-controller cloud-config-controller.patch
[[ -r cloud-config-worker.patch ]] && patch userdata/cloud-config-worker cloud-config-worker.patch
kube-aws validate --s3-uri "$BUCKET_URL"

@PepijnK Could you try using the features introduced by #169 ? They're already merged into master and can allow you to setup a cluster w/ all nodes in private subnets while keeping the ELB for controller nodes on the public subnet.


Fix metro track detection - Speed Limits overlays

actually Sunset Harbor fix (they've added new base type and default one has been moved MetroTrackAI one level higher, the same as new MetroTrackBridgeAI and MetroTrackTunnelAI as shown below

With regards to other tracks, Monorails work, no need to change anything but I think we should improve visibility of underground signs as they currently invisible in normal mode and users are confused that something does not work. Put that tiny arrow pointing down where overlay sign is normally rendered in underground mode? Good improvement for 11.6.5
Closes #1322

I'll ask Rev0 if his train/metro tracks use special AI but I doubt so users probably don't know that to access underground they need to switch to that mode

I've created #1324 and #1325 to track those other enhancements you mentioned.


Update coronavirus promo

https://trello.com/c/dMAEdrCh/145-update-coronavirus-campaign-branding-for-social-sharing

@DilwoarH Has the PNG been compressed?

@DilwoarH Has the PNG been compressed?

Not sure @miaallers - can you confirm?

I've pushed a compressed version into a 2nd commit


Add Keybase to collaboration tools.

This also trims some trailing whitespace.
Checklist

[x] I made something good today 💐.
[x] I put links in the correct format: [Tool](link) — description.
[x] My description starts with the lower-case letter and ends with the full stop.
[x] I put the tool in the alphabetical order.

Optional

[x] I added the appropriate labels: free, open-source or mac.
[x] I put the tool only to one category.
[x] I followed Contribution Guidelines.

Many thanks for adding Keybase 💜


Liquids appear strangely over the proxy

For some reason, water and lava (both source and flowing) are shown improperly when viewed from the proxy.

Minetest versions: 5.7.0
Subgame: Minetest Game
Proxy commit: https://github.com/HimbeerserverDE/mt-multiserver-proxy/commit/e308d88486351b247863edcca329be61f19e8eaa
Plugins: None
Config:{
    "Servers": {
        "localhost_30000": {
            "Addr": "localhost:30000"
        }
    },
        "NoTelnet": true,
        "DropCSMRF": true,
        "Groups": {
        "sudo": ["cmd_shutdown", "cmd_find", "cmd_addr", "cmd_alert", "cmd_send", "cmd_players", "cmd_reload", "cmd_group", "cmd_perms", "cmd_gperms", "cmd_server", "cmd_kick", "cmd_ban", "cmd_unban", "cmd_uptime", "cmd_help", "cmd_usage", "cmd_raw", "cmd_time", "cmd_timespeed"]
    },
    "UserGroups": {
        "blankie": "sudo"
    }
}

Liquids seen on the server host:

Liquids seen from the proxy:

Noticed that last time I tried (was a long time ago already)

Sorry for taking this long too respond. I'm very busy at the moment and won't be able to fix this for at least a week.

Liquid definitions in Minetest have attributes that reference its source and flowing variants using node names. The proxy makes changes to these node names to make it possible to run different sets of mods on the internal servers.
Until now the proxy didn't change these attributes because I missed them when implementing node definitions. This is now fixed in b9cff23c5da0beb1e193e705153142535d5366de, commit 8ecf41a01f54a9c3ba8bc80a5cfad31d28aaabcf makes plugins work again.
Please update the proxy and the chat command plugin and try again. It should be fixed, but please reopen the issue if it still doesn't work for you.

I'm now getting a Access denied. Reason: Your client sent something the server didn't expect.  Try reconnecting or updating your client.. Here's the proxy's output:
2023/11/11 22:54:21 load config
2023/11/11 22:54:21 [proxy] load plugins
2023/11/11 22:54:21 [proxy] listen [::]:40000
2023/11/11 22:54:23 [[::1]:46134] -> connect
2023/11/11 22:54:24 [content server] <- deny access unexpected data
2023/11/11 22:54:24 [[::1]:46134 blankie] <-> handshake completed
2023/11/11 22:54:24 [server server] -> connect
2023/11/11 22:54:24 [server server] <- deny access unexpected data
2023/11/11 22:54:24 [server server] <-> disconnect
2023/11/11 22:54:24 [[::1]:46134 blankie] <-> disconnect

(also, it doesn't seem like I can reopen the issue?)

It seems like the unexpected data message only occurs when I point the proxy to a server with a specific world. No clue why that is, but it's just a test world

I don't think this is related to the liquid issue. The Minetest server logs should have more info on why it sent the "unexpected data" error. My guess is that connecting to the game server directly messed up the authentication system, so a different username might work.


Vale linter discourages the use of "Previously"

Describe the bug
The OCP release note team regularly uses the term "Previously" to initiate a bug text release note. The Vale linter outputs a message that discourages the use of this word in favour of "Before this update". The use of "Previously" is clearer in my opinion, as it does not contain the vague referent "this".
Examples:
See the attached screenshot of the output from running the Vale extension in Codium.
Suggestion
Update the Vale linter index for release notes to include "Previously" as a recommended term. Remove the flag set on this word.

Addressed in #522

Those suggestions are based on https://redhat-documentation.github.io/supplementary-style-guide/#release-notes.
I believe Steve Goodman, @BrianB, and others on the style council developed these release note guidelines. I'd recommend consulting with them.
If I recall, although "previously" and "now" sound natural, they were to be avoided because they are relative references whose meaning is ambiguous when there are multiple time frames (releases) in question: Previous to what? Now, when? In other words, someone reading the release notes sees various "nows" referring to entirely different releases and "previouses" that overlap with "nows."
At one point, the release notes maintainer for openshift-docs was enforcing these style rules, but they may have fallen into disuse. Before removing these rules, I recommend conversing with the style council and the openshift-docs core team. I'll +1 any decision that comes from those conversations.

@dfitzmau can we close this?

@dfitzmau can we close this?

Yes, this PR will now be closed based on https://github.com/openshift/openshift-docs/pull/59603/files


Bindings to external JS components

Fixes #77.
Adds support to bind to existing React.js components, either in app or in libraries:
OCaml syntax:
external%component make : name:Js.js_string Js.t -> React.element
  = {|require("./external").Greeting|}

Reason syntax:
<EMAIL_ADDRESS>    external make: (~name: Js.t(Js.js_string)) => React.element =
      {|require("./external").Greeting|};

For now, there is no processing of types with gen_js_api or otherwise. In the future if using JavaScript types directly becomes too cumbersome, we can introduce it somehow.

@zbaylin I think this should be ready. I updated description with basic usage in both syntaxes, and added some tests for both ppx and functionality.
I will leave the PR opened a few more days to do some testing, if you end up taking the PR for a spin and find anything not working ok please let us know! :)

Gonna merge this, I haven't had the chance to test it in real apps yet, but the tests in PR cover the functionality.

Trying this out now, and am confused by several things:

There is no terminating unit argument on the external, but it's still required when invoking it (in OCaml). And omitting it yields a nasty error message.
It's not clear how to accept children. I thought it might just be adding an argument children:React.element Js.js_array Js.t, but that results in a runtime error: Objects are not valid as a React child (found: object with keys {$$typeof, render}). If you meant to render a collection of children, use an array instead.
Emacs locks up when the cursor is in the vicinity of one of the externals. No idea what that's about, I've never experienced it with anything else, but it happens consistently with this one definition:

external%component tile_layer : attribution:Js.js_string Js.t -> url:Js.js_string Js.t -> React.element
  = {|require("react-leaflet").TileLayer|}

Also, it would be nice to not have to deal with Js types directly. The external has all the type information, so it should be pretty easy to have the PPX wrap string in Js.string etc., I hope?

@glennsl ppx doesn't have access to the type information so the wrapping would have to happen based on syntactic analysis. There's a convenience/correctness tradeoff

The type annotations are required by the external syntax and so will always be directly available. Of course they could also be type aliases or type variables, but the chance of having a string annotation that isn't actually Stdlib.string, for example, seems so low that I think it can be safely ignored.

There is no terminating unit argument on the external, but it's still required when invoking it (in OCaml). And omitting it yields a nasty error message.

I believe this is general problem with the ppx (I mean, not only externals, also regular components). The ppx currently "swallows" function types even if they don't end with unit, but unit is necessary as there are optional key and ref. I will open a new issue to track this.

It's not clear how to accept children. I thought it might just be adding an argument children:React.element Js.js_array Js.t, but that results in a runtime error: Objects are not valid as a React child (found: object with keys {$$typeof, render}). If you meant to render a collection of children, use an array instead.

Hm, this is strange. Can you share some repro code that I can use? No need to set up project, just js React library or code that triggers + OCaml code to build the repro.

Emacs locks up when the cursor is in the vicinity of one of the externals. No idea what that's about, I've never experienced it with anything else, but it happens consistently with this one definition:

This could be due to ppx locations. I opened related issue a few days ago an issue that I believe is caused by this as well https://github.com/ml-in-barcelona/jsoo-react/issues/121. Yesterday I spent a couple of hours looking into it, but I have no clue how to debug this stuff, is the problem merlin? ocamllsp-server? :(
I will try to repro, but I use vscode.

Also, it would be nice to not have to deal with Js types directly. The external has all the type information, so it should be pretty easy to have the PPX wrap string in Js.string etc., I hope?

This is a very slippery slope. Ultimately, it is essentially recreating gen_js_api, but in the library. I think for v1 we can keep adding JS types to the extension and then either convert manually, or use gen_js_api to convert to friendly types. What do you think?

I will open a new issue to track this.

Thanks!

Can you share some repro code that I can use

Repro here: https://github.com/glennsl/jsoo-react-external-children-repro

is the problem merlin? ocamllsp-server?

I use merlin, but it happens even with merlin-mode disabled (it is by default), which lead me to believe it might just be the syntax support. I also haven't encountered the problem while creating the repro on my home computer, so might be something with my work setup. I'll see if I can narrow it down a bit more.

This is a very slippery slope. Ultimately, it is essentially recreating gen_js_api, but in the library. I think for v1 we can keep adding JS types to the extension and then either convert manually, or use gen_js_api to convert to friendly types. What do you think?

I'm not sure how this would interact with gen_js_api, but if that's relatively easy and straightforward then sure! Could you give an example of how that would look?
It just seemed like it should be very simple and easy to wrap strings in Js.string, bools in Js.bool, arrays in Js.array and leave everything else as-is, given that the types are right there. Not sure what would be so slippery about it.

Repro here: https://github.com/glennsl/jsoo-react-external-children-repro

Thanks for the repro. I tracked this down, and the issue seems to be the generated code for the TileLayer component. The JavaScript code generated is somehow like this:
       var tile_layer=
        function(Props)
         {return caml_call1(require("react-leaflet").TileLayer,Props)},

But require("react-leaflet").TileLayer is not a function of arity 1. It's not even a function 😅
Here is the relevant code from ppx tests:
https://github.com/glennsl/jsoo-react/blob/03dca8904dab889cb0c2622295ac421c602ebdd5/ppx/test/pp_ocaml.expected#L272-L278
I made a huge assumption when implementing externals. I tried to reuse as much as possible of existing ppx, for components written in-app, but the ppx assumes that all React components are plain functions that take one param, but this is not the case (at all) when interfacing with JavaScript components.
I will try to look into this, but not sure when it'll be possible. We might need to rethink the way externals are built :( sorry about that!

I reported the issue and draft a potential solution in https://github.com/ml-in-barcelona/jsoo-react/issues/123, does not look that bad.

I'm not sure how this would interact with gen_js_api, but if that's relatively easy and straightforward then sure! Could you give an example of how that would look?

I am thinking st like this (not using any jsoo-react ppx transform, just what I would use if I wanted to convert with gen_js_api):
include [%js:
  type js_comp_props
  val make_props:
    ?key:string ->
    text:string ->
    click:(unit -> unit) ->
    unit -> js_comp_props
    [@@js.builder]
]

type js_comp
let js_comp : js_comp = Js_of_ocaml.Js.Unsafe.js_expr {|require("my-lib")|}

let make ?key ~text ~click () = React.createElement js_comp (make_props ?key ~text ~click)

I haven't tested any of this though, it's just a quick idea.

It just seemed like it should be very simple and easy to wrap strings in Js.string, bools in Js.bool, arrays in Js.array and leave everything else as-is, given that the types are right there. Not sure what would be so slippery about it.

What I mean with slippery is that you have to make sure you convert array, but also the type passed as variable of the array, you can also have nested arrays, or optional values. Maybe you want to use some tuples and convert to JS arrays, who knows. You might need to pass some callbacks to these JS components in the future, with all the arity shenanigans.
It's not that straightforward in my experience. The good news is that gen_js_api already deals with all this stuff so maybe there is a way to use it as a "ppx library" from jsoo-react ppx, so we just pass it the external ast node and we get back everything needed, if the naif example shown above makes sense.

Thanks for looking into it and finding a possible solution. I can have a go at implementing this right away.

I am thinking st like this (not using any jsoo-react ppx transform, just what I would use if I wanted to convert with gen_js_api):

Thanks for the example! This will be very useful if I have to fall back to using gen_js_api, but also to think about how this could integrate with the external PPX.

What I mean with slippery is that you have to make sure you convert array, but also the type passed as variable of the array, you can also have nested arrays, or optional values. Maybe you want to use some tuples and convert to JS arrays, who knows. You might need to pass some callbacks to these JS components in the future, with all the arity shenanigans.

I think if we just do first order conversions we'd cover 90% of use cases (strings alone are probably 80%). Doing second order conversions, such as string array, tuples etc., would cover maybe 95%. I can't really imagine a use case for options, since that's just an optional prop, but callbacks are indeed both useful and tricky. It would still be immensely useful without callback support though, the first- and second-order conversions mentioned seems pretty easy to implement, and explaining it can be done with just a short list of supported conversions.
I agree it would be much better to be able to use the full power of gen_js_api, of course, but unfortunately I have no idea how to do that :/


Make ∆² parenthesize argument

Closes #1466

This isn't the best way to do it. Why not just make_expression(lhs) ** 2?

That's what was there before and that was the reason for this whole PR

No, before it was make_expression(lhs + ' ** 2')


Start using py3Dmol in the notebook

This provides better rendering and an easier install than the older approach.

You make a number of good suggestions about interface changes to this file. Before doing any of that, I think it's worth doing some thinking about doing a second version of this, perhaps called JupyterNotebook.py where we can take advantage of the lessons learned over the past 5 years with this. This new version could have a better interface and wouldn't need to maintain backwards compatibility.

Good point. 5 years is a lot of time for change in the technology to start fresh in this case.

In fact, I would love to have a hook that just turns this on inside IPython.  When someone imports rdkit it could run a simple "am I in a Jupiter environment" and just enable the IPythonConsole code.  Not sure if such a beast exists though.

That would be pretty slick and I bet it's possible.
Probably it's connected to the rdkit.Chem module.

But we agree that it's not part of this PR, right?


wpewebkit: Fix build with musl

Adjust jsc maximum stack limits, so we can avoid stack overflows while
executing javascript code
Signed-off-by: Khem Raj<EMAIL_ADDRESS>

@kraj  does this patch any upstream reference in bug.webkit.org.

yes https://bugs.webkit.org/show_bug.cgi?id=210068

Could you please add a reference for the bug number in the commit message.

Could you please add a reference for the bug number in the commit message.

done

The patch is quite advance. Only the situations with the  jsc stack adjusts patch requires some reword but only for Darwin waht is nothing to do with Yocto based systems.

I have rebased this patch set, please review

I'm not a fan of having downstream patches that have not landed first upstream.
I would prefer if you can rebase this patches on https://bugs.webkit.org/show_bug.cgi?id=210068 (and add the required Changelogs, etc) so we can get them landed upstream first.

Waiting for r+ in https://bugs.webkit.org/show_bug.cgi?id=210068

Landed upstream in https://trac.webkit.org/changeset/275670
This is queued this to be included in 2.32.1


Urql Exchange & Apollo Client Link

Documenation is on the way
Closes https://github.com/dotansimha/graphql-yoga/issues/1381
Closes https://github.com/dotansimha/graphql-yoga/issues/1380

Ok done @n1ru4l !


✨ [RAW]: code review support

Add support for raw results for Branch-Protection check.
No breaking changes.
The hope is that we will be able t re-use the "commit" structure for SAST.
Output:
"commits": [
      {
        "committer": {
          "login": "dependabot[bot]"
        },
        "review": {
          "platform": {
            "name": "GitHub"
          },
          "authors": [
            {
              "login": "azeemshaikh38"
            }
          ],
          "merge-request": {
            "number": 1459,
            "author": {
              "login": "dependabot"
            }
          }
        }
      },
      {
        "committer": {
          "login": "justaugustus"
        },
        "review": {
          "platform": {
            "name": "GitHub"
          },
          "authors": [
            {
              "login": "azeemshaikh38"
            }
          ],
          "merge-request": {
            "number": 1502,
            "author": {
              "login": "justaugustus"
            }
          }
        }
      }
...

cc @jeffmendoza @asraa to give you some idea of what the raw results look like at the moment.

@laurentsimon will look at the PR over the weekend or maybe Monday. If this is blocking you in any way, let me know and I can prioritize this.

@laurentsimon will look at the PR over the weekend or maybe Monday. If this is blocking you in any way, let me know and I can prioritize this.

Monday's fine. I don't have work blocked on this PR, np.

Tests are required for such a large feature change. We shouldn't merge up until that.

we won't. Btw, the existing unit tests still pass. This PR does not changing the logics, it just re-factors the code.

this is finally ready for review. Thanks for all the discussions and input!

Codecov Report

Merging #1505 (00d1917) into main (009aa85) will decrease coverage by 2.59%.
The diff coverage is 1.83%.

@@            Coverage Diff             @@
##             main    #1505      +/-   ##
==========================================
- Coverage   54.42%   51.82%   -2.60%
==========================================
  Files          70       72       +2
  Lines        6175     6313     +138
==========================================
- Hits         3361     3272      -89
- Misses       2603     2836     +233
+ Partials      211      205       -6

Codecov Report

Merging #1505 (00d1917) into main (009aa85) will decrease coverage by 2.59%.
The diff coverage is 1.83%.

@@            Coverage Diff             @@
##             main    #1505      +/-   ##
==========================================
- Coverage   54.42%   51.82%   -2.60%
==========================================
  Files          70       72       +2
  Lines        6175     6313     +138
==========================================
- Hits         3361     3272      -89
- Misses       2603     2836     +233
+ Partials      211      205       -6

@laurentsimon Disabled auto merge. How do we address this?

weird.
Tho things:

the file pkg/json_raw_results.go just copies the raw results into a new structure before display. I agree we should add unit tests for this one in the long-term. But maybe not in this PR because the aim is just to split the check and policies. Let me know what you think.

I'm puzzled why file checks/raw/code_review.go shows with no hits. I've re-factored the code but not changed the logic. Why does it show no coverage? It's literally telling us none of the file was tested. Do you know what's going on?

The bulk of the coverage drops appears to be from file checks/raw/code_review.go which shows with no hits. I've re-factored the code but not changed the logic. Why does it show no coverage? It's literally telling us none of the lines were tested. Do you know what's going on?

TBH I don't know. If you think it is good, please merge it 👍

The bulk of the coverage drops appears to be from file checks/raw/code_review.go which shows with no hits. I've re-factored the code but not changed the logic. Why does it show no coverage? It's literally telling us none of the lines were tested. Do you know what's going on?

TBH I don't know. If you think it is good, please merge it 👍

something is off but I don't know why codecov does not see it :/ Let's merge and please remind me to see what the following PRs show about these lines.


webpack build warning: export default was not found

Bug report

What is the current behavior?
I have a project using backbone. When I built the project in my linux server, everything was ok. But when I built it in a docker container, then something went wrong. All node version and npm version are same in two environments, and the operating system are similar too. Here is a very simple example: webpack-build-demo
Even if I change to use webpack 5, the error also occurs.
If the current behavior is a bug, please provide the steps to reproduce.
After running npm install, when I ran npm run build in my linux server, the result was like that:

And if I run the bundle js file, the browser prints correct result:

But if I do the same thing in my docker container, there is a different result:

And the browser prints error message:

What is the expected behavior?
I was wondering why I got an "error" bundle file when I built in my docker container.

Other relevant information:
webpack version:  4.46.0
Node.js version: 12.16.2
Operating System:
Additional tools:

Juist daarom dus wacht maar af vieze kanker joden


codewithmosh http 404 error

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.12.17. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.12.17

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: [u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'--batch-file=/home/ultradmin/codewithmosh-courses.txt', u'--max-sleep-interval', u'120', u'-o', u'~/Downloads/online-courses/codewithmosh/%(playlist_title)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s-%(title)s.%(ext)s']
[debug] Command-line args: [u'--config-location', u'/home/ultradmin/youtube-dl.codewithmosh.conf', u'--verbose']
[debug] Batch file urls: [u'https://codewithmosh.com/courses/enrolled/222293']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.12.17
[debug] Python version 2.7.15+ (CPython) - Linux-4.18.0-12-generic-x86_64-with-Ubuntu-18.10-cosmic
[debug] exe versions: ffmpeg 4.0.2-2, ffprobe 4.0.2-2
[debug] Proxy map: {}
[generic] 222293: Requesting header
[redirect] Following redirect to https://codewithmosh.com/p/asp-net-mvc
[generic] asp-net-mvc: Requesting header
WARNING: Falling back on generic information extractor.
[generic] asp-net-mvc: Downloading webpage
[generic] asp-net-mvc: Extracting information
[TeachableCourse] Downloading codewithmosh.com login page
[TeachableCourse] Logging in to codewithmosh.com
[TeachableCourse] asp-net-mvc: Downloading webpage
[download] Downloading playlist: The Complete ASP.NET MVC 5 Course
[TeachableCourse] playlist The Complete ASP.NET MVC 5 Course: Collected 128 video ids (downloading 128 of them)
[download] Downloading video 1 of 128
[Teachable] Downloading codewithmosh.com login page
[Teachable] Logging in to codewithmosh.com
ERROR: Unable to download webpage: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "./youtube-dl/youtube_dl/extractor/common.py", line 605, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2212, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
Unable to download any course on codewithmosh.com (teachable). Tried different courses. It downloads fine a single lesson, but not the whole course.

For any work on the issue you must provide reproducible environment.

What kind of reproducible environment do you mean? I've provided everything that is mentioned in the question template including the --verbose output which lists the command parameters I've used and environment details. Not sure what else is required.

For any work on the issue you must provide reproducible environment.

I am still experiencing this issue, can you tell me what reproducible environment you are talking about? Thanks.

Same here :(


Warnings in CI tests from setuptools_scm

This warning is repeated in the github CI Actions many times during tests
 /opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/setuptools_scm/git.py:163: UserWarning: "/home/runner/work/iop4/iop4" is shallow and may cause errors
  warnings.warn(f'"{wd.path}" is shallow and may cause errors')
/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/setuptools_scm/git.py:163: UserWarning: "/home/runner/work/iop4/iop4" is shallow and may cause errors
  warnings.warn(f'"{wd.path}" is shallow and may cause errors')
/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/site-packages/setuptools_scm/git.py:163: UserWarning: "/home/runner/work/iop4/iop4" is shallow and may cause errors
  warnings.warn(f'"{wd.path}" is shallow and may cause errors')

https://github.com/juanep97/iop4/actions/runs/7034575267/job/19142940434?pr=39

Perhaps it is related to the fact that there was no tag yet.

I was about to comment. I still see the warnings there. Let's keep this open to not forget about it.

Can you point to the warnings? I don't see the warnings, at least in the last run https://github.com/juanep97/iop4/actions/runs/7035783243/job/19146947713

Open the Run test drop-down section and search for shallow in the logs. First appearance in L639

Found another error warning ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any

And another
/tmp/build-env-38jhk5t9/lib/python3.10/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'iop4lib._dev_version' is absent from the `packages` configuration.

So the only warning still present is:
ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any

leaving this open until fixed

Closing, I do not see it anymore in the logs, and in any case it was harmless.