simplification

Hi @eduherraiz. I've been looking at ways to simplify the repo / process. This is in line with the list you created.
I have a couple of questions however, at least to make sure I'm not missing anything important, and that I better understand some of the things you have in place already:

I noticed that you use a docker-based pypi server and wheelhouse / --use-wheel. What's the reason for using it? Is it purely for performance reasons when building multiple docker images? (because the initial test build still pulls from pypi as far as I can tell). I'm wondering if the images are all based on one base, then there won't be a real need for this level of caching, because docker will provide it anyway, so that's one thing to potentially simplify. If, however, there's another / different reason, then I'm curious to know.
What's the reason for having test/builder/ Dockerfile.build? Is it purely to "bake" pip packages into the wheelhouse? Or is there another reason?
I can see the reason for using circus inside thumbor-multiprocess, but have you considered / are you open to using native docker/docker-compose scaling with something like nginx-proxy or traefik to handle load balancing across multiple thumbor containers. This has the benefit of simplifying things by having less types of images to build/maintain.

traefik + alpine+python3 based image would be a nice change

The ideas here (and more) were implemented in https://github.com/MinimalCompact/thumbor

Thank you for your comments @gingerlime
The wheelhouse and old build system had the objective to reduce the size of the final image, trying to compile some huge packages and just share compiled with the final image with the wheels. Some of the packages for opencv and others had a lot of big build-dependencies.
In our new build system we removed all of this.
The thumbor-multiprocess image was prepared basically for some old enviroments where we can't use this proxy or orquestration tools. We really encourage to use the thumbor base image in all the moderns enviroments and we will just maintain the multiprocess image for historical reasons.
I'm closing this issue and I'll open another ones if we can take some of your work on the fork.


chore: Add color config for nutrition and food groups

Implements #34
Adds colour config for food groups and nutrition to the tailwind config.

I used the food groups that were specified in the designs. We can and should use food groups that are specified in the data, but in order to do so, we need to:

Provide the food-group names in a way that is compatible with CSS class-name conventions, ie. without white spaces or commas, I suggest we add key field or so that provides this key.
@faustoperez needs to provide the colours for all of these groups.

@faustoperez needs to provide the colours for all of these groups.

These are the colours for the 10 main categories. Note that @cameronkruse suggested that "Sugars and sweeteners" was moved to "Others" and move "Grains" to its own category

Link to Figma file 👉 https://www.figma.com/design/zz7755JriAt4vKJsdV3Dcr/Global-Food-Twin?node-id=978-1093&t=YNlZVM29GFwEfA4A-1

@oliverroick I added a slug property to the food group with a web-friendly string in #44.

@vgeorge I've updated the config based on the discussion here. These new groups still need to be imported in the database.


php artisan db:seed

[ErrorException]
join(): Passing glue string after array is deprecated. Swap the parameters

Same problem

Same problem, any solution?

After doing a bit of research, I found out the error is related to fzaninotto/faker library, which as you may already know it has been archived some time ago. I tried searching all the join() uses and swapping the parameters but I still get the same error on PHP 7.4.
Some people report that doing what I explained above on PHP 7.3 solves the issue. Otherwise you may need to get fake data to seed the Users table from somewhere else and avoid using the Users seed included.

Can't seem to get it working, if you don't mind i'd like to be assigned to this issue. I think I might be able to get it to work.

Very nice research
On Wed, Apr 28, 2021, 21:33 MFernandez93 @.***> wrote:

Can't seem to get it working, if you don't mind i'd like to be assigned to
this issue. I think I might be able to get it to work.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Vuedo/vuedo/issues/54#issuecomment-828758841, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AID7I7RCYJ67P67VJNB74TTTLBWKVANCNFSM4Z7Y4W6A
.

Nice research
On Wed, Apr 28, 2021, 19:25 MFernandez93 @.***> wrote:

After doing a bit of research, I found out the error is related to
fzaninotto/faker library, which as you may already know it has been
archived some time ago. I tried searching all the join() uses and swapping
the parameters but I still get the same error on PHP 7.4.
Some people report that doing what I explained above on PHP 7.3 solves the
issue. Otherwise you may need to get fake data to seed the Users table from
somewhere else and avoid using the Users seed included.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Vuedo/vuedo/issues/54#issuecomment-828679501, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AID7I7TPL43EPUNUKEUGGLLTLBHKPANCNFSM4Z7Y4W6A
.

Well, since the problem lies in the faker library and not with the current codebase, I think this issue can't be fixed with a PR. To properly seed the database I inspected Laravel's logs, until I found the following line:

[2021-04-29 00:06:25] local.ERROR: ErrorException: join(): Passing glue string after array is deprecated. Swap the parameters in /home/manu/proyectos/vuedo/vendor/cviebrock/eloquent-sluggable/src/Services/SlugService.php:133

So I went to SlugService.php line 133 and fixed that specific use of the join() function, after that I was able to run php artisan db:seed

Well, since the problem lies in the faker library and not with the current codebase, I think this issue can't be fixed with a PR. To properly seed the database I inspected Laravel's logs, until I found the following line:

[2021-04-29 00:06:25] local.ERROR: ErrorException: join(): Passing glue string after array is deprecated. Swap the parameters in /home/manu/proyectos/vuedo/vendor/cviebrock/eloquent-sluggable/src/Services/SlugService.php:133

So I went to SlugService.php line 133 and fixed that specific use of the join() function, after that I was able to run php artisan db:seed

You mean this line???
#1 D:\laragon\www\vuedo-master\vendor\cviebrock\eloquent-sluggable\src\Services\SlugService.php(133): join(Array, ' ')
how to fix it ?????

just update your package.
composer update ...


Remove configFile param since it is already the default

liberty-maven-plugin already uses src/main/liberty/config/server.xml as the default location for server.xml.  There is no need to specify it in the pom if just using this default, otherwise it shows warnings such as [WARNING] The /Users/eric/git/SkillsNetworkLabs/src/main/liberty/config/server.xml file is overwritten by the /Users/eric/git/SkillsNetworkLabs/src/main/liberty/config/server.xml file.

Thanks for the PR @ericglau I will merge and test as I do rember that warning showing :)


Maximum call depth easily exceeded in Typst 0.10.0

When using Typst compiler version 0.10.0 (where the maximum call depth was decreased from 256 to 64), any straight line tree with more than 8 rules in a row will cause the maximum call depth to be exceeded.
Example:
#prooftree(
  axiom("1"),
  rule("2"),
  rule("3"),
  rule("4"),
  rule("5"),
  rule("6"),
  rule("7"),
  rule("8"),
  rule("9"),
)

Output of typst c example.typ:
error: maximum layout depth exceeded
 = hint: try to reduce the amount of nesting in your layout

help: error occurred in this call of function `measure`
    ┌─ prooftree.typ:209:22
    │
209 │       let width(it) = measure(it, styles).width
    │                       ^^^^^^^^^^^^^^^^^^^

help: error occurred in this call of function `width`
    ┌─ prooftree.typ:239:30
    │
239 │       let wbranches_nolabel = width(branches) - children_label_wleft - children_label_wright
    │                               ^^^^^^^^^^^^^^^

help: error occurred in this function call
    ┌─ prooftree.typ:112:17
    │
112 │         let elem = rule_func(
    │ ╭──────────────────^
113 │ │         settings,
114 │ │         styles,
115 │ │         stack.slice(stack.len() - to_pop)
116 │ │       )
    │ ╰───────^

I see, currently the library is creating a stack for each level where the bottom element is the bottom-most line and the top element is the rest of the tree, which itself contains a recursive stack etc etc. This means the more layers you add the more nested it becomes, which now more easily triggers the error (you could probably still hit the error before with a tree 4x deeper than yours).
I can see a potential solution by manually calculate where each element should go to and then placing them all at once. It should be feasible, though it would require me to rewrite a large part of the library. I'll see what I can do.

typst 0.11.0 introduces context expression. will this help?

typst 0.11.0 introduces context expression. will this help?

I only gave it a quick look since I didn't have too much time in the last weeks, it looks like it may help a bit, but I'll still need to completly rework how the library draws the tree.

I reworked how the library performs layouting and it no longer produces recursive content trees, which was the source of the error, so this should now be fixed.

🙌 Thank you @SkiFire13


refactor(app): replace Text with StyledText

Overview
This PR will replace Text with StyledText and remove color={COLORS.darkBlack} which is the current
default text color. The files this PR updated are using components from atoms since the files that don't use components from atoms will not be needed anymore or will be updated soon.
fix #11149

Changelog

Text --> StyledText
remove color={COLORS.darkBlack}
do a few housekeeping
ex width={'11rem'} --> width="11rem"
0.5rem --> SPACING.spacing3

Review requests

Text --> StyledText
The changes align with our recent update

Risk assessment
low

This PR replaces {'string'} with "string". The rest will be solved by  #11211.


Race condition in Sunflow benchmark

I found a race condition that can occur when running the sunflow benchmark on multiple threads, resulting in the following stack trace:
===== DaCapo unknown sunflow starting warmup 1 =====
Exception in thread "Thread-1" java.lang.NullPointerException
	at org.sunflow.core.Geometry.intersect(Geometry.java:93)
	at org.sunflow.core.Instance.intersect(Instance.java:135)
	at org.sunflow.core.InstanceList.intersectPrimitive(InstanceList.java:30)
	at org.sunflow.core.accel.BoundingIntervalHierarchy.intersect(BoundingIntervalHierarchy.java:559)
	at org.sunflow.core.Scene.trace(Scene.java:199)
	at org.sunflow.core.LightServer$1.run(LightServer.java:231)
	at java.base@21.0.1/java.lang.Thread.runWith(Thread.java:1596)
	at java.base@21.0.1/java.lang.Thread.run(Thread.java:1583)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:784)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:211)
java.lang.reflect.InvocationTargetException
	at java.base@21.0.1/java.lang.reflect.Method.invoke(Method.java:580)
	at org.dacapo.harness.Sunflow.validate(Sunflow.java:79)
	at org.dacapo.harness.Benchmark.run(Benchmark.java:195)
	at org.dacapo.harness.TestHarness.runBenchmark(TestHarness.java:200)
	at org.dacapo.harness.TestHarness.main(TestHarness.java:153)
	at java.base@21.0.1/java.lang.reflect.Method.invoke(Method.java:580)
	at Harness.main(Unknown Source)
	at java.base@21.0.1/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
Caused by: java.lang.RuntimeException: Image check failed! - #errors: 89833
	at org.sunflow.Benchmark.print(Benchmark.java:252)
	at org.sunflow.system.UI.printError(UI.java:63)
	at org.sunflow.Benchmark.kernelEnd(Benchmark.java:240)
	... 8 more

The issue seems to lie in the sunflow project itself and has laid dormant for years.
Taking a look at the implementation of the Geometry class I was able to find the cause of the issue.
The upshot is in this snippet:
    private AccelerationStructure accel;
    private int builtAccel;

    void intersect(Ray r, IntersectionState state) {
        /* [...] */
        if (builtAccel == 0)
            build();
        accel.intersect(r, state); // This line throws NPE due to accel == null
    }

    private synchronized void build() {
        if (builtAccel != 0)
            return;
        if (primitives != null) {
            int n = primitives.getNumPrimitives();
            /* [...] */
            accel = AccelerationStructureFactory.create(acceltype, n, true);
            accel.build(primitives);
        } else {
            accel = new NullAccelerator();
        }
        builtAccel = 1;
    }

Here intersect is being called concurrently from multiple threads.
The problem is builtAccel is a plain old int that is set in a synchronized method, but read in a non-synchronized method.
Because there's no forced ordering between builtAccel and accel, it is possible for accel to be loaded before builtAccel is (or for builtAccel to be set to 1 before accel is initialized), leading to a NPE.
The upstream fix would be to make builtAccel volatile to force a memory barrier between checking and setting builtAccel. I'm not sure how feasible it would be to patch the benchmark, so a workaround that could be applied in the harness would be to somehow call build() on all Geometry instances in the during scene initialization, before any call to intersect is performed.

Nice catch.   Thank you Carlo.
It's easy to patch sunflow.  I've just done that in 1bb053a.
I was not able to reproduce the race, so if you can do so, could you please test the fix?
Just build sunflow on its own for testing.
ant sunflow


[Question] imports insed less works?

I'm trying to use to import a file that has import. but does not work. it's supported?
styless:
const HeaderStyled = styled(Layout.Header)`
  @import (reference) "tmp.less";
  &.light {
    color: @text-color;
    background: @layout-sider-background-light;
  }

  &.dark {
    background: @layout-header-background;
    color: @layout-trigger-color;
  }
`;

less:
/* tmp.less */
@text-color: red;
@import "test.less";

/* test.less */
@import "test2.less";
@test: red;
.test {
  color: @test;
}

@text-color: cyan;

/* test2.less */
@test: red;
.test2 {
  color: @test;
}

Thanks for the detailed test case, I will add it as a test soon.
Nested imports are supported,
In this example, the selectors .test and .test2 are not injected because tmp.less is imported as a (reference).
Also, the class .dark is not generated since the variables @layout-header-background and @layout-trigger-color are not defined.
Let me know if this helps.

I did not mean to close the defect by merging my test addition.
If you need more information, feel free to re-open it.
Please note that you may need to clean your babel cache when editing the source less files.
I added the command yarn clean to clear the jest cache.
If the variables are meant to be dynamic, I prefer setting them in the theme store rather than a less import. See this for more info.

I will test without (reference) today later.
the variables are not very dynamic, but I'm working on a way to charge them less, supporting the import's with a custom webpack loader:
const less = require("less");
const getOptions = require("less-loader/dist/getOptions");
const lessToJs = require("./less-vars-to-js");

const parseResult = (context, options) => {
  const promise = new Promise(function(resolve, reject) {
    less.parse(context, options, function(err, root, imports) {
      if (err) reject(err);
      else resolve({ root, imports });
    });
  });

  return promise;
};

module.exports = async function(context) {
  const loaderContext = this;
  const options = getOptions(loaderContext);
  let results = [];
  lessToJs;

  const lessToJsOptions = {
    resolveVariables: true,
    stripPrefix: true
  };

  const {
    // root,
    imports
  } = await parseResult(context, options);

  results = Object.keys(imports.contents).map(key => imports.contents[key]);
  results.reverse();

  let lessContent = results.join("\n");

  lessContent = lessContent.replace(/@import.[^;]*;/gm, "");
  const lessJs = lessToJs(lessContent, lessToJsOptions);

  return `module.exports = ${JSON.stringify(lessJs)}`;
};

and
import React from "react";
import { ThemeProvider as StyledThemeProvider } from "styled-components";

/* eslint-disable */
import * as themeDefault from "!!../utils/less-var-loader!./default/index.less";
/* eslint-enable */

import "./index.less";

const theme = {
  ...themeDefault
};

const ThemeProvider = ({ children }) => {
  return <StyledThemeProvider theme={theme}>{children}</StyledThemeProvider>;
};

export default ThemeProvider;

but this don't work properly. I'm thinking about getting the variables from the less tree instead of the parsed string with lessToJs.

I agree that less-vars-to-js is limited as it search for variables in one file only.
The less compiler itself can scan through imports and find all variables.
In your code above, what did you get for themeDefault?

a little sample:
{
  "functions": "~`(function() {\n  var hueStep = 2",
  "blue-1": "color(~`colorPalette(\"@{blue-6}\", 1)`)",
  "blue-2": "color(~`colorPalette(\"@{blue-6}\", 2)`)",
  "blue-3": "color(~`colorPalette(\"@{blue-6}\", 3)`)",
  "blue-4": "color(~`colorPalette(\"@{blue-6}\", 4)`)",
  "blue-5": "color(~`colorPalette(\"@{blue-6}\", 5)`)",
  "slider-handle-color": "color(~`colorPalette(\"@{primary-color}\", 3)`)",
  "slider-handle-color-hover": "color(~`colorPalette(\"@{primary-color}\", 4)`)",
  "slider-handle-color-focus": "tint(@primary-color, 20%)",
  "slider-handle-color-focus-shadow": "tint(@primary-color, 50%)",
  "slider-handle-color-tooltip-open": "#1890ff",
  "slider-dot-border-color": "hsv(0, 0, 91%)",
  "slider-dot-border-color-active": "tint(@primary-color, 50%)",
  "slider-disabled-color": "fade(#000, 25%)",
  "slider-disabled-background-color": "#fff",
  "tree-title-height": "24px",
  "tree-child-padding": "18px",
  "tree-directory-selected-color": "#fff",
  "tree-directory-selected-bg": "#1890ff",
  "collapse-header-padding": "12px 0 12px 40px",
  "collapse-header-bg": "hsv(0, 0, 98%)",
  "collapse-content-padding": "16px",
  "collapse-content-bg": "#fff",
  "message-notice-content-padding": "10px 16px"
}

full: https://gist.github.com/gullitmiranda/9ec9944ee31ce77b3366698fa311dd77

Some values like @slider-handle-color-tooltip-open would work.
However, I am afraid that the more complicated variables @blue-1 will not work.
Pre-parsing with less might resolve those variables.

ye, I'm try without (reference) and thong work:

OBS: i'm updated js-css style to update cache.

const HeaderStyled = styled(Layout.Header)`
  @import "../themes/tmp.less";
  &.light {
    color: @text-color;
    background: @layout-sider-background-light;
  }

@import "test.less";
@text-color: blue;
@layout-sider-background-light: cyan;

/* @import "test2.less"; */

@test: red;
.test {
  color: @test;
}

@text-color: cyan;

never mind, it worked. the cache was not updated as expected.

I am glad it works.
I am still looking at a solution to invalidate the cache automatically. Bebel itself does not handle the cache invalidation, it is the responsibility of the bundler.

now I am having problems with the support of javascriptEnabled less option 😭 (pr in WIP: https://github.com/jean343/styless/pull/31):
Error converting the less syntax for the file: /playground/styless/test/Component/Javascript.test.js `
    @import "import.less";
    color: @echoColor (blue);
    background: @primary-1;
  ` { [Error: JavaScript evaluation error: missing ) after argument list from `echoColor("[props["blue-6"],(props.theme || {})["blue-6"],"#1890ff"].filter(v => v !== void 0)[0]", 1)`]
  message:
   'JavaScript evaluation error: missing ) after argument list from `echoColor("[props["blue-6"],(props.theme || {})["blue-6"],"#1890ff"].filter(v => v !== void 0)[0]", 1)`',
  stack: undefined,
  type: 'Syntax',
  filename:
   '/playground/styless/test/Component/javascript.less',
  index: 324,
  line: 14,
  column: 18,
  callLine: NaN,
  callExtract: undefined,
  extract:
   [ '@primary-color : @blue-6;',
     '@primary-1: color(~`echoColor("@{primary-color}", 1)`);',
     '' ] }

I do want to support as much of the Less language as possible.
I am afraid that javascriptEnabled is deprecated, and I could not find documentation around the ~`` syntax.
I am in favor of passing less arguments to the compiler, and I will approve your PR when complete.
As per your issue, it is transpiling blue-6 to the props syntax, is @echoColor (blue) the correct syntax?

the best syntaxe is @echoColor(blue), and it's work well. the problem is with variables. but already solved this in the PR #31

:tada: This issue has been resolved in version 1.4.13 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Blazor client side

Hi
I am not able to use electron with blazor client side.
Thanks

Hi @cissemy, you should definitely host with ASP.NET Core, then it should work without any problems.
Alternatively, server-side works in any case!


error "The turn cost storage must be initialized only once."

Version: 0.11.0-pre1
In my config: graph.flag_encoders: car|turn_costs=true,bike2,racingbike,mtb,hike
+ ./graphhopper.sh -a import -i /Volumes/Pegasus/maps/australia-latest.osm.pbf -c /Users/develar/Documents/gh-routing-data/config.yml
## using java 1.8.0_172 (64bit) from
## using existing osm file /Volumes/Pegasus/maps/australia-latest.osm.pbf
## existing jar found web/target/graphhopper-web-0.11-SNAPSHOT.jar
## now import. JAVA_OPTS=-Xmx15g -Xms10g -XX:+UseG1GC
INFO  [2018-06-16 06:13:58,777] com.graphhopper.reader.osm.GraphHopperOSM: version 0.11|2018-06-07T15:12:38Z (5,14,4,3,3,2)
INFO  [2018-06-16 06:13:58,782] com.graphhopper.reader.osm.GraphHopperOSM: graph CH|car,bike2,racingbike,mtb,hike|RAM_STORE|3D|turn_cost|,,,,, details:edges:0(0MB), nodes:0(0MB), name:(0MB), geo:0(0MB), bounds:1.7976931348623157E308,-1.7976931348623157E308,1.7976931348623157E308,-1.7976931348623157E308,1.7976931348623157E308,-1.7976931348623157E308, CHGraph|fastest|car, shortcuts:0, nodesCH:(0MB), CHGraph|fastest|bike2, shortcuts:0, nodesCH:(0MB), CHGraph|fastest|racingbike, shortcuts:0, nodesCH:(0MB), CHGraph|fastest|mtb, shortcuts:0, nodesCH:(0MB), CHGraph|fastest|hike, shortcuts:0, nodesCH:(0MB)
INFO  [2018-06-16 06:13:58,825] com.graphhopper.reader.osm.GraphHopperOSM: start creating graph from /Volumes/Pegasus/maps/australia-latest.osm.pbf
INFO  [2018-06-16 06:13:58,826] com.graphhopper.reader.osm.GraphHopperOSM: using CH|car,bike2,racingbike,mtb,hike|RAM_STORE|3D|turn_cost|,,,,, memory:totalMB:10240, usedMB:19
INFO  [2018-06-16 06:14:13,529] com.graphhopper.reader.osm.OSMReader: creating graph. Found nodes (pillar+tower):14 388 392, totalMB:10240, usedMB:294
INFO  [2018-06-16 06:14:13,541] com.graphhopper.reader.dem.CGIARProvider: cgiar Elevation Provider, from: http://srtm.csi.cgiar.org/SRT-ZIP/SRTM_V41/SRTM_Data_GeoTiff, to: /Users/develar/graphhopper-master/cgiar, as: MMAP_STORE using calcmean: false
INFO  [2018-06-16 06:17:37,511] com.graphhopper.reader.osm.OSMReader: 42 917 671, now parsing ways
INFO  [2018-06-16 06:18:03,549] com.graphhopper.reader.osm.OSMReader: 46 281 931, now parsing relations
INFO  [2018-06-16 06:18:03,667] com.graphhopper.reader.osm.OSMReader: finished way processing. nodes: 2314621, osmIdMap.size:14409849, osmIdMap:180MB, nodeFlagsMap.size:26378, relFlagsMap.size:36001, zeroCounter:21023 totalMB:10240, usedMB:3541
INFO  [2018-06-16 06:18:06,226] com.graphhopper.reader.osm.OSMReader: time pass1:14s, pass2:232s, total:247s
INFO  [2018-06-16 06:18:06,228] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: start finding subnetworks (min:200, min one way:200) totalMB:10240, usedMB:494
INFO  [2018-06-16 06:18:07,238] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: car findComponents time:1.0101064, size:20883
INFO  [2018-06-16 06:18:07,923] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: 381449 subnetworks found for car, totalMB:10240, usedMB:1095
INFO  [2018-06-16 06:18:08,982] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: bike2 findComponents time:1.0588369, size:31280
INFO  [2018-06-16 06:18:09,682] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: 120789 subnetworks found for bike2, totalMB:10240, usedMB:1573
INFO  [2018-06-16 06:18:10,720] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: racingbike findComponents time:1.0376482, size:31403
INFO  [2018-06-16 06:18:11,289] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: 123286 subnetworks found for racingbike, totalMB:10240, usedMB:2050
INFO  [2018-06-16 06:18:12,281] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: mtb findComponents time:0.99158376, size:31207
INFO  [2018-06-16 06:18:12,864] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: 119960 subnetworks found for mtb, totalMB:10240, usedMB:2526
INFO  [2018-06-16 06:18:13,801] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: hike findComponents time:0.9365595, size:20454
INFO  [2018-06-16 06:18:14,393] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: 80641 subnetworks found for hike, totalMB:10240, usedMB:2980
INFO  [2018-06-16 06:18:14,612] com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks: optimize to remove subnetworks (381449), unvisited-dead-end-nodes (364429), maxEdges/node (9)
INFO  [2018-06-16 06:18:14,955] com.graphhopper.reader.osm.GraphHopperOSM: edges: 3 049 022, nodes 2 253 040, there were 381 449 subnetworks. removed them => 61 581 less nodes
java.lang.AssertionError: The turn cost storage must be initialized only once.
	at com.graphhopper.storage.TurnCostExtension.init(TurnCostExtension.java:60)
	at com.graphhopper.storage.BaseGraph.<init>(BaseGraph.java:143)
	at com.graphhopper.storage.GraphHopperStorage.<init>(GraphHopperStorage.java:83)
	at com.graphhopper.util.GHUtility.newStorage(GHUtility.java:306)
	at com.graphhopper.GraphHopper.postProcessing(GraphHopper.java:854)
	at com.graphhopper.GraphHopper.process(GraphHopper.java:671)
	at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:640)
	at com.graphhopper.http.GraphHopperManaged.start(GraphHopperManaged.java:71)
	at com.graphhopper.http.cli.ImportCommand.run(ImportCommand.java:40)
	at com.graphhopper.http.cli.ImportCommand.run(ImportCommand.java:28)
	at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:87)
	at io.dropwizard.cli.Cli.run(Cli.java:78)
	at io.dropwizard.Application.run(Application.java:93)
	at com.graphhopper.http.GraphHopperApplication.main(GraphHopperApplication.java:46)

How do I can enable turn costs for car?

From https://github.com/graphhopper/graphhopper/blob/master/docs/core/turn-restrictions.md I see "Currently, it is not possible to use turn restrictions with the 'speed mode' (contraction hierarchies)", so, probably error message should be improved and explanation comment in the example config modified to make clear that you should use landmarks instead of CH.

It should be possible to use the config. For speed mode the turn costs will still be ignored (until #270 is fixed)
This is indeed a bug if additionally the configuration graph.do_sort: true is used.

Thanks, indeed, problem in the "graph.do_sort: true". The same error if use landmarks and only car.

So turn costs currently are ignored everywhere or just in car/fast profile?

They are ignored if CH is used. If you disable CH they'll be used.

Ok

What does "Ok" mean? Yes or no? Sorry, I am a bit confused.

It means ok, I'm interested :)


Disable debug build of unit tests.

The shared library unit tests are currently failing on FreeBSD when
they're built in debug mode, so this disables the recently added debug
unit test build so that the autotester passes again. The debug unit test
build should be re-enabled once the problem with FreeBSD is fixed.
https://auto-tester.puremagic.com/show-run.ghtml?projectid=1&runid=90669&dataid=625858
The release unit test build was failing on FreeBSD 9 and 10 already, but the autotester is on 8 still, where the release unit test build seems to pass, but it looks like they all fail the unit tests in the debug build (I've verified that with 10 and I assume that 9 does, since 8 clearly does given the failure on the autotester and the fact that 10 fails). It looks like the failing tests are all in the tests for shared library support, so my guess is that the shared library support for FreeBSD has definite issues on all 3 versions. It's just that the issues don't trigger as many test failures with 8, since the unit test release build works on 8 but not 9 or 10. But without further investigation, we obviously don't know exactly what's wrong. Regardless, we clearly can't leave the debug unit test build enabled right now, since it causes the autotester to fail on FreeBSD.
Pull# 1317 added a debug build for the unit tests, and this does not undo the changes that it had to do to the makefiles to make that work, but it does make it so that the debug unit test build is not built as part of the normal unit test build.

Auto-merge toggled on

Will check ASAP.


added all-coins-39-information-user_data to userdata restApi

this is returns the  api call  documented at
https://binance-docs.github.io/apidocs/spot/en/#all-coins-39-information-user_data
All Coins' Information (USER_DATA)

Get information of coins (available for deposit and withdraw) for user.

the response is a java List.   it will always be a filter-loop kind of api call so i have not spent too much time tracking down the schema/docs that weren't included upstream either.
I've written companion code for this patch, it's kotlin but mostly straight streams compatible loops/filters as mentioned above.
https://gist.github.com/jnorthrup/502c883fafc89a0c33d28671c2fbdedb
ALGO(Algorand) : BNB,BTC,BUSD,PAX,TUSD,USDC,USDT
ALICE(My Neighbor Alice) : BTC,BUSD,USDT
ALPHA(Alpha Finance Lab) : BNB,BTC,BUSD,USDT
AMB(Ambrosus) : BNB,BTC,ETH
ANKR(Ankr) : BNB,BTC,PAX,TUSD,USDC,USDT
ANT(Aragon) : BNB,BTC,BUSD,USDT
APPC(AppCoins) : BNB,BTC,ETH
AR(Arweave) : BNB,BTC,BUSD,USDT
ARDR(Ardor) : BNB,BTC,ETH,USDT

unintended push here.


Episode length shorter then chunk length, can't pad the chunk

Hi!
I stumbled across another issue. I try to pad sequence in chunk_sequence function when it is too short to create chunk from it (sequence length < chunk length). I also return shorter chunk length for the last chunk if it is shorter (in this particular code it is always the only chunk, but I wrote it for general case anyway). I saw masking in overshooting calculation so I thought it should work, but there are some infinite values in divergence loss (if I understand it correctly). Here is the code and console out (it is debug config with intentionally set chunk length to too big value):
Code
def chunk_sequence(sequence, chunk_length, randomize=True, num_chunks=None):
  """Split a nested dict of sequence tensors into a batch of chunks.

  This function does not expect a batch of sequences, but a single sequence. A
  `length` key is added if it did not exist already. When `randomize` is set,
  up to `chunk_length - 1` initial frames will be discarded. Final frames that
  do not fit into a chunk are always discarded.

  Args:
    sequence: Nested dict of tensors with time dimension.
    chunk_length: Size of chunks the sequence will be split into.
    randomize: Start chunking from a random offset in the sequence,
        enforcing that at least one chunk is generated.
    num_chunks: Optionally specify the exact number of chunks to be extracted
        from the sequence. Requires input to be long enough.

  Returns:
    Nested dict of sequence tensors with chunk dimension.
  """

  with tf.device('/cpu:0'):
    if 'length' in sequence:
      length = sequence.pop('length')
    else:
      length = tf.shape(nested.flatten(sequence)[0])[0]
    needs_padding = tf.constant(False)

    if randomize:
      if num_chunks is None:
        num_chunks = tf.maximum(1, length // chunk_length - 1)
      else:
        num_chunks = num_chunks + 0 * length
      used_length = num_chunks * chunk_length

      needs_padding = length < used_length
      pad_len = used_length - length
      # TODO: Assert here that padding will be smaller then one chunk length
      sequence = tf.cond(needs_padding,
                         lambda: nested.map(
                             partial(_pad_tensor, length=pad_len, value=1), sequence),
                         lambda: sequence)
      max_offset = tf.cond(needs_padding,
                           lambda: 0 * length,
                           lambda: length - used_length)
      offset = tf.random_uniform((), 0, max_offset + 1, dtype=tf.int32)
    else:
      if num_chunks is None:
        num_chunks = length // chunk_length
      else:
        num_chunks = num_chunks + 0 * length
      used_length = num_chunks * chunk_length
      max_offset = 0
      offset = 0

    clipped = nested.map(
        lambda tensor: tensor[offset: offset + used_length],
        sequence)
    chunks = nested.map(
        lambda tensor: tf.reshape(
            tensor, [num_chunks, chunk_length] + tensor.shape[1:].as_list()),
        clipped)

    chunks['length'] = tf.cond(needs_padding,
                               lambda: tf.concat(
                                   [chunk_length * tf.ones((num_chunks - 1,), dtype=tf.int32),
                                    (chunk_length - pad_len) * tf.ones((1,), dtype=tf.int32)], 0),
                               lambda: chunk_length * tf.ones((num_chunks,), dtype=tf.int32))
    tf.print("Requested length: ", chunk_length, " True length", chunks['length'])
    tf.print("Rewards: ", chunks['reward'])
    tf.print("Actions: ", chunks['action'])

    return chunks

Output
[...] means that out from above repeats in mostly unchanged form.
Rewards:  [[-0.1 -0.1 -0.1 ... 1 1 1]]
Requested length:  110  True length [100]
Actions:  [[[0.773366451 0.66181761 -0.936789095 ... 0.923150301 -0.953526735 0.0622075424]
  [-0.590437293 -0.892674208 0.174977049 ... -0.939423203 -0.186107203 -0.910980761]
  [-0.504322708 -0.61423862 -0.569634855 ... 0.924559832 -0.295185775 0.788345039]
  ...
  [1 1 1 ... 1 1 1]
  [1 1 1 ... 1 1 1]
  [1 1 1 ... 1 1 1]]]
[...]
Caused by op 'graph/CheckNumerics_5', defined at:
  File "/usr/local/anaconda3/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/anaconda3/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/scripts/train.py", line 129, in <module>
    tf.app.run(lambda _: main(args_), remaining)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/scripts/train.py", line 129, in <lambda>
    tf.app.run(lambda _: main(args_), remaining)
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/scripts/train.py", line 102, in main
    for unused_score in run:
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/training/running.py", line 199, in __iter__
    for value in self._process_fn(self._logdir, *args):
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/scripts/train.py", line 87, in process
    training.define_model, dataset, logdir, config):
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/training/utility.py", line 160, in train
    score, summary = model_fn(data, trainer, config)
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/training/define_model.py", line 95, in define_model
    os_posterior, os_mask, config.free_nats, debug=config.debug)
  File "/Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/training/utility.py", line 211, in compute_losses
    losses[key] = tf.check_numerics(loss, key) if debug else loss
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 817, in check_numerics
    "CheckNumerics", tensor=tensor, message=message, name=name)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): divergence : Tensor had Inf values
	 [[node graph/CheckNumerics_5 (defined at /Users/piotr/Projects/Planning-in-Imagination/src/planet/planet/training/utility.py:211)  = CheckNumerics[T=DT_FLOAT, _class=["loc:@graph/optimizer_main/gradients/graph/CheckNumerics_5_grad/CheckNumerics"], message="divergence", _device="/job:localhost/replica:0/task:0/device:CPU:0"](graph/Mean_5)]]

Do you have any hints that could possibly shorten my struggle? 😄
Thanks for help!

Note that we're not using padding in PlaNet but instead randomly crop sub sequences of length config.batch_shape[1] from the collected episodes. If your collected sequences are at least of this length, this might work out of the box. I haven't tested training on variable length sequences though, so it's not a supported feature but if it doesn't work you might get it to work with minor changes.

It is exactly what I'm trying to do. I'll try to root cause it on my own then.

Good luck, hope your project goes well!

@piojanu I looked into this a bit and parallel data collections seems to be working now. You can try this via --params '{collect_batch_size: 5}'.

By parallel you mean batched collection in the collection step right? Not training and collecting in parallel?

Yes, exactly.

Also, in the paper you've written that "we [...] add one additional episode to the data set every C update steps.". With batched collection you then add e.g. 5 new episodes in the collection step?

Yes, although this was not used in the paper.


Commits counting is not correct

https://github.com/go-gitea/gitea, master has 7991 commits, last commit id is 3c0e6d11267bc9c45e6ff58b8c7a8511f4ccb0ab

https://gitea.com/gitea/gitea_mirror is a mirror of above project, master has 7988 commits, last commit id is also 3c0e6d11267bc9c45e6ff58b8c7a8511f4ccb0ab

When use git command

This maybe caused by commits counting cache when the repositories are mirrors, the cache hasn't been cleared.
@lafriks

Could be that there is problem on mirroring

Cannot reproduce it in a recent release. Just close it.


Compartion of GAEZ Data Viewer and pyAEZ data result.

Hi,
After calculate the start date of irrigation and rainfed date, I found in GAZE data viewer also has Beginning date of longest component LGP (day-of-year), but how could I compare the data with my own data? The viewer only provided one result, how could I distinguish the irrigation condition and rainfed condition?
Regards,
Xinyue Chang

Hi @WeirdCxychen,
I don't understand why you compare PyAEZ crop calendar date with GAEZ's longest LGP's beginning date because the concepts behind these two outputs are different.
In LGP concepts, there can be more than one periods of growing period. GAEZ's longest LGP's beginning date only selects the longer periods amongst the available periods in a single year, and the first DOY of that longest period will be longest LGP's beginning date.

PyAEZ's crop calendar, as I mentioned in previous messages, is the DOY the model suggest to start sowing so that at the end of the crop cycle, you would obtain the maximum attainable yield. The PyAEZ model uses LGP to decide whether there's sufficient number of days to simulate biomass and yield.
Sincerely,
Swun Wunna Htet

HI, @NokuzaSezo97
Thank you so much for your illustration. Could you please provide the references to the picture?

@WeirdCxychen,
You could check out this online FAO resource: https://www.fao.org/3/W2962E/w2962e00.htm
Best regards,
Swun Wunna Htet

Thank you so much!
Happy Chinese New year!


Lack of --threads option

As stated in title, would be nice if there was a box like the Delay or Page Range one but for threads, since it would speed up downloading a lot at once like someone's favorites or a specific category/tag.
Maybe add a tip to increase delay to prevent getting timed out for sending too many requests?
Nice tool so far, though - helped with the panic of the site shutting down.

#5 Added threads


Server side file upload

Completed the server side file upload. Currently does not make any changes to the database (need to talk to Alex about this) and does not restrict file type as there seems to be a check already in the system. Also about those unit tests...........

Thanks for the feedback Wassim, one question though, when there isn't an "uploads" directory in somebody's project nothing actually saves and an error occurs, is there any way to safeguard against this without tracking the folder in git?

Just fixed the styling issue as well as changed the gitignore.
P.S. I also added a hidden placeholder file to the uploads folder(since git doesn't track empty folders)


Added doc for Browserless 1click

Started adding the missing docs for the services and created a cover template.

Thank you for the PR!


Python3.6 and installation

Can you explain please how to run a new version of mitmproxy on Debian 9? When I run
"mitmproxy" I always get the error, that python 3.6 or higher must be installed, despite the fact that I've installed it.
Any other comments? What have you tried so far?

Aliases for "python" and "python3" pointing to python3.6
update-alternatives switching to python3.6

3."python --version" says "Python 3.6.4"

Installing python 3.6 from sources and from testing repository
Installing mitmproxy from git
Installing mitmproxy via pip

System information
Debian-9-x86_64
Mitmproxy 4.0.3

If this can help someone - the kind of solution is to download the source code of mitmproxy and run "python3.6 setup.py install" - so in this way mitmproxy will be linked to python3.6. But I don't think it's the right solution, and surely this is not the easiest way of installation. So still waiting for another variant.

How did you install mitmproxy? With apt-get? with pip3?
Apt-get won't work at all.
If you use pip3, make sure to use the one from python3.6 - and not the system one! You can check with pip3 --version

I installed mitmproxy with pip3. As I've checked recently pip3 --version is saying that it was installed into python3.5 folder. So how should have I installed pip3 to have it being installed into python3.6 folder? I was just followig instructions from here https://docs.mitmproxy.org/stable/overview-installation/.

sudo apt install python3-pip
sudo pip3 install -U pip
sudo pip3 install mitmproxy

Well yes - Debian 9 simple does not provide a Python version that is compatible with mitmproxy - thats why we put in the warning/error message.
You need to install Python3.6 including pip3 for that python version, then you can use pip3 install mitmproxy. I suggest you take a look at https://github.com/pyenv/pyenv

Can you explain please how to install Python3.6 including pip3? I installed python3.6 from here https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tgz and if I am not mistaken it doesn't include pip. When I run command from your instructions (sudo apt install python3-pip ) pip is linked to the python3.5 (because there is no python3.6 in stable version of Debian). So the question now is how to install pip to have it working with python3.6 by default? I think you should update your instruction after you have moved to new python version which is not even supported in stable versions of Debian.

Our docs clearly state that we don't support older Debian versions. There is no point in explaining something we don't support.
Did you look at the pip documentation on how to install it? https://pip.pypa.io/en/stable/installing/

Excuse me, Debian 9 is the latest stable version for now. The new Debian 10 which supports python3.6  is only in testing version. So you claim that you don't support Debian at all?

Not at the moment, yes. This is stated in our docs: sudo apt install python3-pip # Debian 10 or higher, Ubuntu 17.10 or higher

So you claim that you don't support Debian at all?

You can of course use the precompiled binaries or Docker - we just don't support this specific installation method on Debian 9. There's probably a way to install mitmproxy from source nonetheless, but we don't have the resources to look into it - sorry.

Did you look at the pip documentation on how to install it? https://pip.pypa.io/en/stable/installing/
Yes, now I've found how to install it manualy, but I thought there must be an easier way like it was when mitmproxy worked with python3.5. Now I see that there isn't. Anyway, I think that you have to mention in your instructions how to install pip3.6 or python3.6  on older versions of Debian and other Linux versions. I can't even call the Debian 9 "older" because it's the latest stable version. I think the same situation with Ubuntu. If you do it, I think the list of your users will be wider. For example, I had no problems of installing mitmproxy 3.0.4 and eairlier, and updating it. It was done just by few commands. But when you upgraded to python3.6 I stucked with it.

Anyway, thank you for help guys. You are doing right job.

We did not take the decision lightly to move to Python 3.6.
Here is a comprehensive list of Linux variants that provide py3.6 directly:
https://github.com/mitmproxy/mitmproxy/issues/2266
As mentioned in our docs, pyenv is an easy way to install new Python versions.
Here are a few commands that should be straightforward (untested):
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
# now do what pyenv-installer told you to do
pyenv install 3.6.5
pyenv shell 3.6.5
pip3 install mitmproxy

Not that complicated after all, is it?


standard model with resolution of 64x64x64

Hi, could you provide the skip concat model with output resolution of 64x64x64 ? Thank you.

@KHao123 Unfortunately I don't think I have pure .pt model in my hand (I have to find out). However I do have various file such as ONNX, TFlite, and mlmodel please email me if you need further models.


Schema field options not getting a callback with ui-material

Vulcan 1.15
Any callback placed in options at a schema's field will result in an app crash with ui-material.
Example
Vulcan-Starter, locale on Users, it gets:
options: () => Locales.map(({ id, label }) => ({ value: id, label })),
This crashes material-ui's Select component with options being undefined

When you say "crash", I assume there must be some kind of error message?

Uncaught TypeError: properties.options is not iterable
at SelectComponent (Select.jsx:8)
This would fix the issue for example:
options: Locales.map(({ id, label }) => ({ value: id, label }))

Investigated a bit, this does not seem related to Material UI. Using options as a function will also fail in Bootstrap, eg try the example-forms and change "categories" options to a function in packages/example-forms/lib/modules/customers/schema.js.

@SachaG this is probably an issue the recent changes, but I can't tell when options was called previously, in the case where it's a function and not directly an array

Progressing, the relevant commit is: "6c8d214f70c0152be5f39bb2239796cd20ad527e" "Do not handle options at form level anymore since we don't have field-specific data yet".
You removed the code that handled option as functions from form, which makes sense when the data are not yet there, but at this point I can't tell where its used. It is supposed to be called by some inner component? It seems to fail the Bootstrap Select and Mui Select at least

It seems that this new behaviour is already handled correctly in "Autocomplete", but just not in Checkbox groups, radios, and select at this point.
As far as I understand, the packages/vulcan-forms/lib/components/FormComponentLoader.jsx will also do some of the work. But it is only present if the user specify a field query explicitely.

Sorry, I found the issue. We just need to add this in FormComponent:
    // if there is no query, handle options here; otherwise they will be handled by
    // the FormComponentLoader component
    if (!this.props.query && typeof this.props.options === 'function') {
      fciProps.options = this.props.options(this.props);
    }

Like you said I split the logic in two, for when there is a query and when there isn't. But forgot to replace the options() function call for the second scenario…

@yairtal I pushed the fix to devel, can you let me know if it works for you?

Works fine on my end with both ui-bootstrap and ui-material, thank you!


[SPARK-43817][SPARK-43702][PYTHON] Support UserDefinedType in createDataFrame from pandas DataFrame and toPandas

What changes were proposed in this pull request?
Support UserDefinedType in createDataFrame from pandas DataFrame and toPandas.
For the following schema and pandas DataFrame:
schema = (
    StructType()
    .add("point", ExamplePointUDT())
    .add("struct", StructType().add("point", ExamplePointUDT()))
    .add("array", ArrayType(ExamplePointUDT()))
    .add("map", MapType(StringType(), ExamplePointUDT()))
)
data = [
    Row(
        ExamplePoint(1.0, 2.0),
        Row(ExamplePoint(3.0, 4.0)),
        [ExamplePoint(5.0, 6.0)],
        dict(point=ExamplePoint(7.0, 8.0)),
    )
]

df = spark.createDataFrame(data, schema)

pdf = pd.DataFrame.from_records(data, columns=schema.names)

spark.createDataFrame()
For all, return the same results:
>>> spark.createDataFrame(pdf, schema).show(truncate=False)
+----------+------------+------------+---------------------+
|point     |struct      |array       |map                  |
+----------+------------+------------+---------------------+
|(1.0, 2.0)|{(3.0, 4.0)}|[(5.0, 6.0)]|{point -> (7.0, 8.0)}|
+----------+------------+------------+---------------------+

df.toPandas()
>>> spark.conf.set('spark.sql.execution.pandas.structHandlingMode', 'row')
>>> df.toPandas()
       point        struct        array                   map
0  (1.0,2.0)  ((3.0,4.0),)  [(5.0,6.0)]  {'point': (7.0,8.0)}

Why are the changes needed?
Currently UserDefinedType in spark.createDataFrame() with pandas DataFrame and df.toPandas() is not supported with Arrow enabled or in Spark Connect.
spark.createDataFrame()
Works without Arrow:
>>> spark.createDataFrame(pdf, schema).show(truncate=False)
+----------+------------+------------+---------------------+
|point     |struct      |array       |map                  |
+----------+------------+------------+---------------------+
|(1.0, 2.0)|{(3.0, 4.0)}|[(5.0, 6.0)]|{point -> (7.0, 8.0)}|
+----------+------------+------------+---------------------+

, whereas:

With Arrow:

Works with fallback:
>>> spark.createDataFrame(pdf, schema).show(truncate=False)
/.../python/pyspark/sql/pandas/conversion.py:351: UserWarning: createDataFrame attempted Arrow optimization because 'spark.sql.execution.arrow.pyspark.enabled' is set to true; however, failed by the reason below:
  [UNSUPPORTED_DATA_TYPE_FOR_ARROW_CONVERSION] ExamplePointUDT() is not supported in conversion to Arrow.
Attempting non-optimization as 'spark.sql.execution.arrow.pyspark.fallback.enabled' is set to true.
  warn(msg)
+----------+------------+------------+---------------------+
|point     |struct      |array       |map                  |
+----------+------------+------------+---------------------+
|(1.0, 2.0)|{(3.0, 4.0)}|[(5.0, 6.0)]|{point -> (7.0, 8.0)}|
+----------+------------+------------+---------------------+

Spark Connect

>>> spark.createDataFrame(pdf, schema).show(truncate=False)
Traceback (most recent call last):
...
pyspark.errors.exceptions.base.PySparkTypeError: [UNSUPPORTED_DATA_TYPE_FOR_ARROW_CONVERSION] ExamplePointUDT() is not supported in conversion to Arrow.

df.toPandas()
Works without Arrow:
>>> spark.conf.set('spark.sql.execution.pandas.structHandlingMode', 'row')
>>> df.toPandas()
       point        struct        array                   map
0  (1.0,2.0)  ((3.0,4.0),)  [(5.0,6.0)]  {'point': (7.0,8.0)}

, whereas:

With Arrow

Works with fallback:
>>> spark.conf.set('spark.sql.execution.pandas.structHandlingMode', 'row')
>>> df.toPandas()
/.../python/pyspark/sql/pandas/conversion.py:111: UserWarning: toPandas attempted Arrow optimization because 'spark.sql.execution.arrow.pyspark.enabled' is set to true; however, failed by the reason below:
  [UNSUPPORTED_DATA_TYPE_FOR_ARROW_CONVERSION] ExamplePointUDT() is not supported in conversion to Arrow.
Attempting non-optimization as 'spark.sql.execution.arrow.pyspark.fallback.enabled' is set to true.
  warn(msg)
       point        struct        array                   map
0  (1.0,2.0)  ((3.0,4.0),)  [(5.0,6.0)]  {'point': (7.0,8.0)}

Spark Connect

Results with the internal type:
>>> spark.conf.set('spark.sql.execution.pandas.structHandlingMode', 'row')
>>> df.toPandas()
        point         struct         array                    map
0  [1.0, 2.0]  ([3.0, 4.0],)  [[5.0, 6.0]]  {'point': [7.0, 8.0]}

Does this PR introduce any user-facing change?
Users will be able to use UserDefinedType.
How was this patch tested?
Added the related tests.

merged to master


DisplayServer: Fix registration of GetRenderingDriversFunction

Fixes segfault running godot --help as it couldn't retrieve the valid rendering drivers.

Excerpt from the --help output:
  --audio-driver <driver>          Audio driver ['PulseAudio', 'ALSA', 'Dummy'].
  --display-driver <driver>        Display driver (and rendering driver) ['x11' ('vulkan')].
  --rendering-driver <driver>      Rendering driver (depends on display driver).

I changed lists of platform-specific drivers (and tests) to use square brackets instead of parentheses, so that they're easier to distinguish from explanatory parentheses.


Unable to validate the information provided

I have followed the document instruction to add the email address attribute, but it was unsuccessful. It returns the error below:

Is this something you need to change in the instructions?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4ace5e69-e1db-52c5-9c57-026fc6145a84
Version Independent ID: f6fcdaac-449a-5a37-5fc8-bef1ed731a2a
Content: Add user attributes and customize user input - Azure AD B2C
Content Source: articles/active-directory-b2c/configure-user-input.md
Service: active-directory
Sub-service: b2c
GitHub Login: @msmimart
Microsoft Alias: mimart

@gxgrammatikos  Thank you for your query . We will investigate and update the thread.

@gxgrammatikos There seems to be some misconfiguration in your policy files.
Why are you capturing the email address twice during the sign-up? I don't see the send verification code button for the email verification. Have you disabled it by setting EnforceEmailVerification to false, as mentioned below?
<ClaimsProvider>
  <DisplayName>Local Account</DisplayName>
  <TechnicalProfiles>
    <TechnicalProfile Id="LocalAccountSignUpWithLogonEmail">
      <Metadata>
        <Item Key="EnforceEmailVerification">false</Item>
      </Metadata>
    </TechnicalProfile>
  </TechnicalProfiles>
</ClaimsProvider>

Hi,
I tried enabled and disabled, but this is not the case.
The problem is that I’m getting the error message when clicking the button Create.
From: AmanpreetSingh-MSFT @.>
Sent: Wednesday, September 22, 2021 3:35 PM
To: MicrosoftDocs/azure-docs @.>
Cc: George Chrysovalantis Grammatikos @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] Unable to validate the information provided (#81421)
@gxgrammatikoshttps://github.com/gxgrammatikos There seems to be some misconfiguration in your policy files.
Why are you capturing the email address twice during the sign-up? I don't see the send verification code button for the email verification. Have you disabled it by setting EnforceEmailVerification to false, as mentioned below?

Local Account

<TechnicalProfile Id="LocalAccountSignUpWithLogonEmail">

  <Metadata>

    <Item Key="EnforceEmailVerification">false</Item>

  </Metadata>

</TechnicalProfile>

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/81421#issuecomment-924887962, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSZVDHFL465KERWQ6H2LKTUDHEPNANCNFSM5EQ4L2OA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@gxgrammatikos can you share your LocalAccountSignUpWithLogonEmail technical profile?

Hi ,
Please, find attached the profile files.
From: AmanpreetSingh-MSFT @.>
Sent: Wednesday, September 22, 2021 3:44 PM
To: MicrosoftDocs/azure-docs @.>
Cc: George Chrysovalantis Grammatikos @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] Unable to validate the information provided (#81421)
@gxgrammatikoshttps://github.com/gxgrammatikos can you share your LocalAccountSignUpWithLogonEmail technical profile?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/81421#issuecomment-924894815, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSZVDBEKXE7LFW2TORGKRDUDHFSNANCNFSM5EQ4L2OA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@gxgrammatikos Attachment is missing.

Could you please check now?
From: AmanpreetSingh-MSFT @.>
Sent: Wednesday, September 22, 2021 3:57 PM
To: MicrosoftDocs/azure-docs @.>
Cc: George Chrysovalantis Grammatikos @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] Unable to validate the information provided (#81421)
@gxgrammatikoshttps://github.com/gxgrammatikos Attachment is missing.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/81421#issuecomment-924903641, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSZVDBCDJH3HH5UFO5PZP3UDHHAPANCNFSM5EQ4L2OA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@gxgrammatikos I don't think gitub allows attachments via email

How can I send it?
From: AmanpreetSingh-MSFT @.>
Sent: Wednesday, September 22, 2021 4:17 PM
To: MicrosoftDocs/azure-docs @.>
Cc: George Chrysovalantis Grammatikos @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] Unable to validate the information provided (#81421)
@gxgrammatikoshttps://github.com/gxgrammatikos I don't think gitub allows attachments via email
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/81421#issuecomment-924921658, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACSZVDDXXCJ4LVQW3L4GPMLUDHJNVANCNFSM5EQ4L2OA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

profiles.zip

@gxgrammatikos Thanks for provided required info.
Looking at the policy files, I can see that you have defined a new claim for email address as . However, you need to either use the built-in claim, which is Email, or use the emailAddress as extension attribute (ref to https://docs.microsoft.com/en-us/azure/active-directory-b2c/user-flow-custom-attributes?pivots=b2c-custom-policy).
The example attribute "City" in this document is the built-in attribute (Azure AD B2C built-in attributes), whereas emailAddress is not a built-in attribute and must be defined as extension attribute like .
Refer to the Important note provided in the document:


Wrong CDC class ID

Describe the bug
I'm trying to setup a STM32F769 as USB device implementing a Virtual COM port with USBX.
I'm having issues on the device as it doesn't perform as expected right after the enumeration is complete.
I've traced down the problem to the CDC class id being wrong.
According to USB docs the class ID should be 0x02.
In USBX this is defined as 10 here which seems to be wrong.
As soon as I change it to 0x02, the check here is successful and the code continues happily.
To Reproduce
Expected behavior
Impact
Prevents the application from working.
Logs and console output
N/A
Additional context
N/A

Thanks for the feedback.
Are you using CDC-ACM class with IAD or without IAD?
If your framework descriptor does not include IAD can you try to add IAD and check if it works with the previous definition? Thanks.

@xiaocq2001 thanks for looking into this.
I've tried a descriptor with and without IAD. The outcome was the same...

CDC-ACM combines two interfaces, the first interface is control and class ID is 2, the second one is data and class ID is 10, in USBX device stack the IAD is checked to associate class driver to both CDC-ACM interfaces, the data interface is checked to obtain endpoints to communicate. This have been working for long time.
As far as I know, ST provided USBX device CDC-ACM example, can you check that as reference?

Yes, this is the code I've been using for reference: https://github.com/STMicroelectronics/x-cube-azrtos-f7/tree/main/Projects/STM32F769I-Discovery/Applications/USBX/Ux_Device_CDC_ACM.
I can confirm that I can run the project successfully on their STM32F769I-Discovery from the STM32 Cube IDE.
The reason for mentioning this is that my project is related with .NET nanoFramework: I'm adding support for USBX in our official Azure RTOS distribution, for the STM32 platform.
The board I'm using it's a custom one, based on the same STM32F769I part, just that it doesn't have the USB PHY as the STM32F769I-Discovery has. Could it be that it can explain the "misbehave" I'm seeing here?

From your description, it seems the same code (original example) has different behavior on different board. It's better to get actual flow captured to see what's the difference to analyze.


Log location is incorrect

The last paragraph mentions the log location, this doesn't match the standard log location which should be:
C:\Program Files\Microsoft Azure Recovery Services Agent\Temp

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59156518-1b9d-d0bb-5dea-6a88e39150ef
Version Independent ID: 3a4f2440-52eb-91a0-3efb-09b3c74dd736
Content: Troubleshoot slow backup of files and folders in Azure Backup
Content Source: articles/backup/backup-azure-troubleshoot-slow-backup-performance-issue.md
Service: backup
GitHub Login: @genlin
Microsoft Alias: genli

Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

@jessehouwing， thanks for pointing out the problem. I will update the article.


   This story is to generate fake HL7s too.
This includes the ability to generate HL7s that represent both incoming data (eg careevolution) and outgoing (most states).
This will involve understanding how our current HL7 code uses (or does not use) schemas.   On the outgoing side, much of our HL7 code uses parametric in the receiver setting yaml configuration.    I don't know how incoming hl7 gets its schema layout.   We may need to separate generating incoming HL7 into a different task?

This is worth doing under the context of HCA and FHIR in the hole.   Needs grooming.

Hey team! Please add your planning poker estimate with ZenHub @jimduff-usds @smythe5 @whytheplatypus @oslynn @carlosfelix2 @jh765

Delete in favor of first creating a plan for testing then creating the issues per the AC of #4138


Potentially confusing prompts when enabling Exposure Notifications

Current Implementation

User may face unexpected / confusing questions about Location permissions when turning on Exposure Notification. Just a heads up - I was not able to test it with Corona Warn App yet but the same happens on both Latvian and Italian apps which are Exposure Notification-based, it seems to be triggered by Google Services and not by country-specific apps.

Suggested Enhancement

User may get a heads-up from Corona Warn App that he has to agree to turn on Location service, and can disagree with "Improving location accuracy". Not sure why the last prompt appears, perhaps it will be patched by Google since it is not necessary for the app to work properly. I was able to reproduce 3 prompts on Android 6 but on Android 10 only first 2 prompts appear.
French app gives following heads-up to the user (it has to get standard "Location" access though, but from user perspective it does not differ much from prompts above):

Important message: Attention, Android will ask you for access to this phone's location. StopCovid will only use Bluetooth detection of nearby phones, and never location data

Expected Benefits

May be possible to increase number of users agreeing to first two prompts (which is necessary for the app to work), and assure users that sharing location data with Google is not necessary and they can safely disagree without decreasing app functionality.

Internal Tracking ID: EXPOSUREAPP-2890

This issue is related to "Misleading / incorrect instruction to "Allow location access" during onboarding" https://github.com/corona-warn-app/cwa-app-android/issues/1541 where I brought up the topic that there are static texts stating that location must be enabled. If #1541 is resolved, then the information should become dynamic and correspond to the current settings and needs of the app, depending on Android version.

Hi Kamil @kbobrowski !
I'm not sure that the app can do much more than it is already doing in current versions like 1.14.3.
The app outputs the following informational texts during onboarding:

The pop-up "Turn on the device location setting" will be shown if the location setting is disabled in Android settings during onboarding and if the Android version is 10 or less. This message is output by the Google Exposure Notifications System, and the app doesn't have control of this message.
The pop-up "Improve location accuracy?" I'm seeing also on Android 8 (I don't have a physical Android 6 device available to test on.) This pop-up isn't shown by Android 10. That corresponds with your findings. In any case it is a function of the Android system, and again the app has no control of the message.
 The location issue doesn't seem to be causing confusion these days. At least it is not coming up as new issues here on GitHib.

I suggest closing this issue. CWA 1.15.1 shows the following messages during onboarding, which are now much clearer.

The other messages are from the operating system and the CWA app has no control over them.

Hi @kbobrowski, see https://github.com/corona-warn-app/cwa-app-android/issues/1721#issuecomment-808451893. We suggest closing this issue. Best, wishes, DS

Corona-Warn-App Open Source Team


Half overlay

fixes #548 :
When hovering over a segment with the lane arrow tool only half of the segment is highlighted letting the user know which half is being modified.
fixes #555: I took the opportunity to fix this related issue.
I Removed buggy/redundant/weird code .
node highlights have the average radius of all connected segments see https://github.com/krzychu124/Cities-Skylines-Traffic-Manager-President-Edition/issues/555#issuecomment-557764120
EDIT by aubergine: Just adding image of the 'half overlay' so reviewers know what to look for:

Found one small issue (with normal lane arrow tool selected):

That half of the segment does not contain any outgoing lanes (it's the 'incoming end' of a one way road).
While clicking has no effect, it would be better if the overlay was not displayed when the segment half has no outgoing lanes (ie. tool not applicable).

Found one small issue (with normal lane arrow tool selected):
While clicking has no effect, it would be better if the overlay was not displayed when the segment half has no outgoing lanes (ie. tool not applicable).
Edit: Alternatively, it could show red 'half overlay' instead of the usual blue color. Not sure which would be best approach (if tool not applicable: no overlay, or different color)?

Wow great observation! This was already the problem without my half segment highlights. Its a Good idea to fix it as part of PR.
I like the Idea of the red overlay too. currently I already use red for when segment selected and ALT is down in my previous commit. So I think I need to change that too.
if (altDown && HoveredSegmentId == SelectedSegmentId)
    color = MainTool.GetToolColor(true, true);

Looks orange to me (when Alt is held down while hovering segment, or Ctrl is held down while hovering a node).

With this update I see only the highlighted half of the segment gets updated when Alt+Click which is much better.
I wonder if it would be worth adding that toggle feature I discussed in #548? So Alt+Click toggles the lane between turning lane or vanilla lane depending on it's current state.

Possible bug with Alt+Click: I'm on a left hand traffic (LHT / RHD) map, so this turning lane choice seems wrong (it would cut across oncoming traffic):

Although maybe I'm not understanding it correctly?
In UK, the dedicated turning lane is usually the one of the left, with the lane to the right going ahead+right. Here are some examples...
At a junction:

At a roundabout (where 'turning lane' means 'you must exit roundabout at next junction'):

We even cut up bus lanes to make room for turning lanes:

I suspect it may be different in other countries (ie. not purely dependent on the driving side). As such, a mod option may be beneficial at some point? (would have to think carefully how to word it)

Further to my comment above about which side should get the turning lane, I noticed that if there are three lanes to choose from vanilla automatically chooses these lane arrows:

And in that situation an Alt+Click does turn the left lane in to a turning lane (which seems correct):

(sorry, I know this is getting off topic of the PR, but thought I'd mention it just in case)

Maybe when holding Alt key, we should highlight the half-lane that will be affected (make it really obvious to use which lane will change), rather than the half-segment?

Maybe when holding Alt key, we should highlight the half-lane that will be affected (make it really obvious to use which lane will change), rather than the half-segment?

I am not sure if that is practical. The calculations that needs to be done in each game click will be too much (unless if i cache it I guess). In another PR maybe.
Mean while I have done this:
Pictures below respectively:
hover
ALT hover
hover (or ALT hover) with no outgoing lanes
selected segment
selected segment + ALT hover

hmmm. I am going to change the color of alt hover to be strong blue and different from selected segment.

I ended up changing the alpha:
hover:

ALT hover:

@aubergine10 Sorry I missed some of your messages

Looks orange to me (when Alt is held down while hovering segment, or Ctrl is held down while hovering a node).

Not any more. I already fix this.

I wonder if it would be worth adding that toggle feature I discussed in #548? So Alt+Click toggles the lane between turning lane or vanilla lane depending on it's current state.

I suspect it may be different in other countries (ie. not purely dependent on the driving side). As such, a mod option may be beneficial at some point? (would have to think carefully how to word it)

In the last PR we talked about state machine. I created #567 to address it.

Possible bug with Alt+Click: I'm on a left hand traffic (LHT / RHD) map, so this turning lane choice seems wrong (it would cut across oncoming traffic):

Nice catch! that is a bug. I think its more fitting to fix it in #567

Damn it I have been using LHD in my code to refer to British roads

I regularly make similar mistake, confusing the four terms: LHT, RHD, vs. RHT, LHD
See my comment in #567 regarding 'nearside' and 'farside' terminology which is driving direction agnostic.

After playing around with the highlights I can still spot some caveats. Please put this review on halt until I fix them. I will continue the discussion about colors and other stuff in #548.
So far I have the following problems:

I want to improve the colors again
Hovering highlights half segment but selection highlights the hole thing. this is inconsistent.
null exception occures when I select one segment but ALT-hover over another.
The length of the highlighted segment is not always half the segment, this can confuse the user because when he hovers the mouse over the would-be highlighted area of the segment, the highlight does not come up.

null exception occurs when I select one segment but ALT-hover over another (possibly #563).

I'm not sure it is as user is reporting the exception when save loads or, if not then, when saving a game. Also, we've had similar error reported (see #566) for TM:PE STABLE v10.20 which doesn't even contain these features. I suspect something in base game or steam common distro has changed which is triggering some of the errors reported.
That being said, good catch on the null exception when hovering other half of selected segment!

@kvakvs @aubergine10 @krzychu124
I sorted out everything. My code should be perfect :)
I am ready for the next round of review.

Note how quickly features get Alt, Shift etc modifiers. We must quickly come with a solution for UX, users will forget these keys or never know they exist.

A bit weird segment highlight when outgoing segment is too short, but other than that works good.

A bit weird segment highlight when outgoing segment is too short, but other than that works good.

Whenever a segment has lane arrows at both ends, only one half of it will get highlighted so the user can see which end of segment is being altered if they click.
If the segment only has arrows at one end, then whole segment is highlighted (it doesn't matter which end you click on). This was done to make it easier to click on short segments when they only have arrows at one end.

@aubergine10  Yes, I know, but it still looks weird.
When I'm hovering over some node, I'm wiling to change arrows of incoming lanes to this node, not somewhere else.
Maybe I'm too meticulous, but I see inconsistency there, because both segments are outgoing. First on the right is highlighting on red(OK, there are no lane arrows at both ends) but the other is fully highlighted blue when it shouldn't be highlighted at all near that node in my opinion.

Ah, I see - the other end of that short node in your GIF doesn't actually have lane arrows, despite being "junction node", because it's 1-way street. Good catch!

Yes, both segments are 1-way streets, but different hover behavior, which might annoy user and provoke missclicks - you can't click on red highlighted segment (OK) but on that weird blue, sure, then lane arrows window will show up near the other node.

@krzychu124

Yes, both segments are 1-way streets, but different hover behavior, which might annoy user and provoke missclicks - you can't click on red highlighted segment (OK) but on that weird blue, sure, then lane arrows window will show up near the other node.

Good catch. I will fix it. :)

Would it be worth removing the red highlight also?
For color vision impaired users, distinguishing between red and blue might be difficult (I'd have to read through my notes on #287 to check).
Maybe instead of red highlight, just don't highlight at all. That way it's simple UX: If it highlights, you can click it to do something, otherwise it's not relevant to the selected tool.

@aubergine10

For color vision impaired users, distinguishing between red and blue might be difficult (I'd have to read through my notes on #287 to check).

Good idea! Makes my job easier too :) . In any case I will fix the problem @krzychu124  raised because fixing it also makes my code shorter.
One more case. What if we have a junction with one way out. changing lane arrows don't make any sense as there is only one direction to go to. So maybe we shouldn't highlight it?
I don't know how to take U turn into consideration.
That said it is possible to have lane arrows toward wrong direction for example :
step 1- manually override lane arrows to go to a road.
step 2- delete that road.

in such circumstances you can use lane arrow tool to fix the arrows.
It would be nice if we could automatically remove lane arrows toward lane direction. Also we should not allow the user to put lane arrows toward wrong directions.
So talking all that into consideration do you think I should allow selection of cases where there is only one way out?

One more case. What if we have a junction with one way out. changing lane arrows don't make any sense as there is only one direction to go to. So maybe we shouldn't highlight it?

Agreed. Possibly related issues: #344

I don't know how to take U turn into consideration.

U-turns are a mess at the moment, there's like 2 or 3 ways to do them (off the top of my head: Lane connectors, and junction restrictions). We should only have one way to implement u-turns. I think there's already some issues tracking that. Possibly related issues: #364, #347, #365, #293

That said it is possible to have lane arrows toward wrong direction for example :

Yup, this is a general problem with all the tools. We need to have a think as to how to handle that situation better and, in particular, make sure the user realises their road changes just screwed up their customisations.  Possibly related issues: #368, #457, #43
Will test this PR again later this evening if time permits.

Should train tracks be excluded? They have no visible lane arrows so any changes made would be invisible to user, which would result in difficult to diagnose problems with rail routing. I suspect this is more general issue with the lane arrows tool.

Same with tram tracks:

And monorail:

And metro:

There's some flags (#513 for example) that indicates what vehicles can use the network, so you could filter to just those that allow cars?

Regarding that stuff above about filtering out non-car networks, that could be a separate PR if desired as it's general issue of the lane arrow tool and not specific to the work you've been doing.
The remaining comments from @krzychu124 are more important for this PR.

@aubergine10

The remaining comments from @krzychu124 are more important for this PR.

Yes that one is fixed already.

Regarding that stuff above about filtering out non-car networks, that could be a separate PR if desired as it's general issue of the lane arrow tool and not specific to the work you've been doing.

Good idea. This PR is already growing out of control. Created #585.
If there are no bugs in this code lets merge this in before the code gets out of date.

I am ready for the next round of this PR


Included big endian example in RAW opening dialog

Include a numpy dtype example for big endian raw data in the RAW file opening dialog. Fixes #144
This commit includes fixes in setup.py to allow building the JS client on Windows. Credit goes to @sk1p !

Looks good on Linux too, merging!


Flutter Session Recordings (Android/iOS)

Description
https://github.com/PostHog/posthog/issues/12344#issuecomment-1421007621
People also asked on the slack community channel.
Related issue https://github.com/PostHog/posthog/issues/13267 and https://github.com/PostHog/posthog/issues/12344
Related Flutter issues
https://github.com/flutter/flutter/issues/129098
https://github.com/flutter/flutter/issues/66863
https://github.com/flutter/flutter/issues/117382

Do you think we can already use it in a Web environment via:
https://posthog.com/docs/session-replay/installation#add-to-your-website-and-product
If we add this script can we already got session record in Web ?
Thanks,

Do you think we can already use it in a Web environment via: posthog.com/docs/session-replay/installation#add-to-your-website-and-product
If we add this script can we already got session record in Web ? Thanks,

That has to be tested, I am unsure, it might work for the html renderer but not for canvaskit I guess.
Maybe the canvaskit renderer requires canvas support which is in BETA yet.
Would you like to give it a try?

I'm using rudderstack to handle posthog, see https://github.com/rudderlabs/rudder-sdk-flutter/issues/151 for more info.

I was able to use session recording, with the JS configuration. But I got some white background on the session record event with the Capture canvas elements enabled.
Using flutter build web --web-renderer canvaskit  or flutter build web --web-renderer html change nothing. So I see some click/textinput but with with background. (so can't see the text white etc...).

@Kiruel can you share your JS configuration?
@daibhin would be interested in checking this out.

@Kiruel which version of the JS SDK have you tried? there was a bug fixed in this version, not sure if related though.

@Kiruel I would certainly like to look into this more for you. Could you send a report using the in-app support form with a link to the problematic recording and I can take a closer look.

I'm using the SDK rudderstack, to send session recording. With the JS config for web env. I can send you all information in DM (slack?) or in the in-app form you send me.

I just send you the Ticket 10418. Thanks.

@daibhin I can also reproduce this locally.
https://us.posthog.com/project/41706/replay/018dd001-6a18-7c6b-9b1b-e6f2559e32be
Just run flutter run -d chrome --web-renderer canvaskit on this folder https://github.com/PostHog/posthog-flutter/tree/main/example

It's taken me a while to investigate this properly. It looks like Flutter wraps the canvas in a custom HTML element. I believe we are not correctly capturing the children of that element and hence ignoring the canvas. Seeing if there is a way around that, two ideas I'm exploring:

Allowing a different root to be set so we can skip the custom elements in the tree (https://github.com/PostHog/posthog-js/pull/1061). Not my favourite solution to expose that because we haven't needed it before now
Seeing if there is a way for rrweb to capture custom elements. Will dig on that a little further...

Reopened because only the Flutter web session recording is supported right now.
This issue also tracks Android and iOS session recording support.

I believe that implementing a screen recording feature is not that complicated. The problem is doing this based on the listeners of the user's actions in the Flutter element tree. You would have to implement a track on the widget clicks and base the recording life cycle on that.
What is your idea? @marandaneto

I believe that implementing a screen recording feature is not that complicated. The problem is doing this based on the listeners of the user's actions in the Flutter element tree. You would have to implement a track on the widget clicks and base the recording life cycle on that.
What is your idea? @marandaneto

Yes, but I wonder if we can leverage the Android and iOS SDK first (via native), using screenshot mode, so all happens in native, if possible.
The screen clicks, network, console logs, etc we can do through the Flutter bits on Dart.

I believe that implementing a screen recording feature is not that complicated. The problem is doing this based on the listeners of the user's actions in the Flutter element tree. You would have to implement a track on the widget clicks and base the recording life cycle on that.
What is your idea? @marandaneto

Yes, but I wonder if we can leverage the Android and iOS SDK first (via native), using screenshot mode, so all happens in native, if possible. The screen clicks, network, console logs, etc we can do through the Flutter bits on Dart.

Yes, it is possible to take advantage of native screen captures. We would just have to have a channel that communicates with Flutter's clicks to the native listeners. I'll try to make a demo to see if this really works.

Over at Sentry, they are using toImage on a renderobject to get a screenshot.
https://github.com/getsentry/sentry-dart/pull/2209

Hi there! We're using the PostHog Flutter SDK in our Mobile app and were wondering if there's an estimated timeline for when Session Recordings might be available? Any insights would be greatly appreciated. Thanks!

I am in the same situation as @Bruno-Meowtel. We are using PostHog in web and mobile app and we are very interested in to have this feature for mobile apps.

We are also looking for session replays for Flutter! Currently, when enabled we see a white screen for web recordings, with the cursor moving around and clicking. @Bruno-Meowtel @jonastg

We are also looking for session replays for Flutter! Currently, when enabled we see a white screen for web recordings, with the cursor moving around and clicking. @Bruno-Meowtel @jonastg

@swxdigital please raise a new issue instead since this is about Android and iOS, and please share more context such as a recording link, browser, if you have canvas enabled in the project settings, etc.
cc @daibhin

Are there any updates on session replays for flutter mobile? I would suggest to give this feature a higher priority, since this looks to be in progress since a year now.

Are there any updates on session replays for flutter mobile? I would suggest to give this feature a higher priority, since this looks to be in progress since a year now.

@ndrslmpk The work started at the beginning of this quarter and it should have an alpha version within the next 2-3 weeks.
I don't think we have announced this as a working in progress before, just that it was planned, unless you have seen this somewhere? Happy to fix any misinformed deadlines.

Did the alpha launch? It's there a timeline for the beta?

Yes please! Would love to be a beta tester. Thank you!

Docs PR
The link will be this one as soon as it's merged.
For the beta tester, please just follow the instructions, install, run it, and let us know if you face any issues.
Please create new issues for that with detailed info such as the OS, the version, the Flutter version, the SDK version, the link of a bad recording, etc.
Thanks a bunch.


Combined PDF generation takes long and generates a very big pdf

Since we are using mkdocs for documenting our software and generating a pdf out of that, we noticed that is takes very long for generating the combined PDF.
The final PDF is currently 333 Pages long and could grow even more. Is there any customization that can be done to reduce the time it takes to create the PDF (and also the usage of the memory?)
Also the combined PDF is very large (around 9MB), but there is only mostly text involved. Could we reduce the size by changing the resolution or the compression?

@lukasgeiter could you maybe help out with this?

@alexander-matthiesen Sorry, I don't have the time to look into this.


Android Marshmallow Integration issue

Is this module functioning for android 6. I tried to install followed all the instructions .Build was successful but no registrations happened.Moreover there is no way to confirm that registration was successful. No error was thrown stating the same.

Hi KarthickCSK, the Android module was developed quite long time ago so it might not work well with Android 6. We'll check and get back to you later.

Thanks . Meanwhile i will send you the log details. where i am facing error.So it wil speed up your process

Error 1 :
error: method does not override or implement a method from a supertype
@Override
^
1 error
:react-native-azurenotificationhub:compileReleaseJavaWithJavac FAILED
Error 2:
Manifest merger failed : Attribute application@icon value=(@mipmap/ic_launcher) from AndroidManifest.xml:17:7-41
is also present at [com.microsoft.azure:azure-notifications-handler:1.0.1] AndroidManifest.xml:13:9-45 value=(@drawable/ic_launcher).
Suggestion: add 'tools:replace="android:icon"' to  element at AndroidManifest.xml:13:5-42:19 to override.

Hi @KarthickCSK, I've just merged a PR that fixes error 1. Error 2 is #13, the temporary workaround is removing the android icon tag.

Thanks, but the app is still not working . I found a detail in the log file.
Failed to complete token refresh
java.lang.IllegalStateException: Default FirebaseApp is not initialized in this process com.azurernnotifications. Make sure to call FirebaseApp.initializeApp(Context) first.
Please look into the above issue i tried to get around but no luck


[flutter_tools]fix typo in printHowToConsumeAar,which case sync fail

Description
according to: Groovy Language Documentation:

Any Groovy expression can be interpolated in all string literals, apart from single and triple-single-quoted strings.

I change typo ' to ", and modify related tests.

This is fine. Did you intend to fix a typo or change " for '?

This is fine. Did you intend to fix a typo or change " for '?

yes! ' will not work actually.

[PR Triage] I suspect Linux customer_testing is failing because the PR needs to be rebased. @vimerzhao could you please try rebasing this PR on to of master to retrigger the presubmit checks? Thanks!


Seems to be broken?

This plugin doesn't seem to censor purple drops anymore. My last two purples showed in chat while this was enabled.

I believe this should be resolved in the update. Let me know if you experience it again


[develop] consistency check RO

Cherry-picked https://github.com/MinaProtocol/mina/pull/13777/
Explain your changes:
*
Explain how you tested your changes:
*
Checklist:

[ ] Dependency versions are unchanged

Notify Velocity team if dependencies must change in CI

[ ] Modified the current draft of release notes with details on what is completed or incomplete within this project
[ ] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[ ] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[ ] All tests pass (CI will check this if you didn't)
[ ] Serialized types are in stable-versioned modules
[ ] Does this close issues? List them

Closes #0000

!ci-build-me

!ci-build-me


Tutorial: call to env does not display message in prompt

Going through the tutorial, the call to env (line 39) did not surface a message within or before displaying the prompt. Only a floating input with no clear indication if something was happening, video attached.
https://user-images.githubusercontent.com/2998575/117172796-c0b6ca00-ad91-11eb-8f0c-f743a4f0b970.mp4

Thanks @tbremer . I'll get that sorted today. There have been some minor API changes since I made the tutorial template and I just hadn't updated some of the properties.

Thanks @johnlindquist! Let me know if I can help in any way.