barplot bug if no or one plot will be available

There is a small bug when plotting only one sequence plot beacause the rownames will be deleted as soon as one row is left...
see comment in function grada_plot()

was replaced and is no continued with other difficulties:
#36


Added toggle method

Thanks, @mitchheddles: makes sense, and it's a simple addition.

Published in 2.1.0.


error  Ant Design CSS-in-JS

client.js:2 Warning: [Ant Design CSS-in-JS] Error in ConnectModal: You seem to be using hashed animation 'ripple 3s infinite ease-out', in which case 'animationName' with Keyframe as value is recommended. Selector: :where(.css-dev-only-do-not-override-1okl62o).ant-web3-connect-modal-body | .ant-web3-connect-modal-link-panel | .ant-web3-connect-modal-ripple

fixed in https://github.com/ant-design/ant-design-web3/pull/859


Learning Curve: Training set percentage -> Number of instances

Why does the x-axis in our learning curve plots have "Training set percentage" instead of "Number of instances" or "Number of observations"? I think we should change that.
TODOs:

[x] Rename (see https://github.com/compstat-lmu/lecture_i2ml/commit/fc9fe26805d2afd824fb2c1c8a4ba24652a2924d)
[ ] Make sure that numbers on x-axis fit with Name, e.g. opt$n = opt$percentage * 1000

@berndbischl any objections?

slides-evaluation-learningcurves doesn't exist anymore => Not sure what to do here?


Chitchat Project Broken

⚠ Before posting a new issue, please make sure that your issue hasn't already been resolved in another issue.
Describe the bug
By Default Botfront opens in chitchat project. This error appears on the chitchat dialog page /project/chitchat-YFju788Pq/dialogue
Any way I can force it to open over the created project. also, why this error appears
To Reproduce
Steps to reproduce the behavior:

Go to 'http://SERVERIP/project/chitchat-YFju788Pq/dialogue'
See error

TypeError: Cannot read property 'length' of undefined     at eval (/imports/ui/components/stories/StoryGroupTree.jsx:1)     at ja (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at ss (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at cs (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at fs (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at Js (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at Xs (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at Vs (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at vs (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at Mo (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:49)     at 4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:1849     at 4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:1849     at t._compute (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:1815)     at t._recompute (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:1815)     at Object.o._runFlush (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:1815)     at a (4af93d4b12a5dcc1d1ba3a4fab165bd8d0ef43b7.js?meteor_js_resource=true:1)
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Does this happen on a new project you just created ? What happens if you go on http://SERVERIP/project/bf/dialogue ?

No, it works fine with the created project, trained successfully

Hello @MatthieuJnon . What do you think cause of this error


Templating - Added get_state_changed, get_state_updated, get_state_attr

Description:

get_state_changed('device_tracker.paulus') will return time object of the time the state last changed.
get_state_updated('device_tracker.paulus') will return time object of the time the state or attributes were last updated.
get_state_attr('device_tracker.pauls', 'battery') will return value for a specific attribute of an entity.

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#3724
Checklist:
If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.github.io

If the code does not interact with devices:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[x] Tests have been added to verify that the new code works.

@balloob I originally wanted to use trigger.entity_id.last_change within an automation to calculate the time a window was open with trigger.to_state.last_change as the start value. In my opinion the trigger value is a very powerful tool, but it lecks some functionallity.
For get_state_updated I figured if I have a use case for get_state_changed someone might have for updated. Lastly get_state_attr could be useful when you just want to log a states attribute and don't want to check for a specific value (mostly relevant for int | float values).
Regarding the check if the entity exists: You're right, the template would be failing. I just added the checks to prevent errors crashing Home Assistant.

I don't think that these functions are adding any value. These helpers are not going to help avoiding errors because None is returned and passed to a render function that will blow up.
Besides that, the state objects and their attributes are already available.


ACME sanity check

Right not a failing ACME is kind of confusing:

2021/01/05 14:13:04 admin.mick-ci.com: obtaining certificate: [admin.mick-ci.com] Obtain: [admin.mick-ci.com] solving challenges: admin.mick-ci.com: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/108447721/7139455689) (ca=https://acme-v02.api.letsencrypt.org/directory)

The reason for this is that admin.mick-ci.com exists, but doesn't end up on the boringproxy server since I haven't modified the Varnish proxy yet.
It might be better to check for this first, before it even attempts to set up ACME. Something like $(curl http://admin.domain:7000/test)" == "boringproxy". This was a clearer message can be presented, and it avoids hitting in to ACME rate limits.

There's actually still some higher level design work that needs to be done here. One problem with the current system is that any user on a boringproxy server instance can create a tunnel for any domain pointed at that instance. This is a security vulnerability. For example, imagine if I point *.example.com at the demo instance. Another user could create tunnels for any subdomain on example.com. Even worse, if I point the apex example.com at the instance, another user may be able to hijack it before I set up my tunnel.
I've considered a couple solutions to this:

Require users to add special DNS TXT records to tie domains to their specific boringproxy account. In this case boringproxy would have to do a DNS check to verify a proper TXT record before creating the tunnel.
Require users to add domains from the boringproxy interface before using them.

I've been leaning towards option 2, because it enables you to integrate boringproxy directly with DNS provider APIs. So not only can it set up tunnels, it can handle DNS for you so you don't have to log in to your DNS. You could even integrate domain purchasing. This is part of the original vision for boringproxy. However in practice (for me at least) I rarely need to change DNS settings, so I'm not sure it's worth the added complexity to boringproxy. To really be secure with option 2, you'd probably still need some level of DNS integration (either TXT records or API key) to ensure that the boringproxy user has control over the domain.
@arp242 thoughts?

There's actually still some higher level design work that needs to be done here. One problem with the current system is that any user on a boringproxy server instance can create a tunnel for any domain pointed at that instance. This is a security vulnerability. For example, imagine if I point *.example.com at the demo instance. Another user could create tunnels for any subdomain on example.com. Even worse, if I point the apex example.com at the instance, another user may be able to hijack it before I set up my tunnel.
I've considered a couple solutions to this:

Require users to add special DNS TXT records to tie domains to their specific boringproxy account. In this case boringproxy would have to do a DNS check to verify a proper TXT record before creating the tunnel.
Require users to add domains from the boringproxy interface before using them.

I've been leaning towards option 2, because it enables you to integrate boringproxy directly with DNS provider APIs. So not only can it set up tunnels, it can handle DNS for you so you don't have to log in to your DNS. You could even integrate domain purchasing. This is part of the original vision for boringproxy. However in practice (for me at least) I rarely need to change DNS settings, so I'm not sure it's worth the added complexity to boringproxy. To really be secure with option 2, you'd probably still need some level of DNS integration (either TXT records or API key) to ensure that the boringproxy user has control over the domain.
@arp242 thoughts?

But people would need access to the server in order to do that, right?
I'm not sure if this is really a security issue in boringproxy as such, but rather in the ACME protocol: it assumes that if you have access to whatever a domain points at, you have rightful control over that domain. I think that's mostly a reasonable assumption. Personally I wouldn't worry too much about this, if I understand the issue correct.
The big downside of the dns-01 verification is that you need the modify the DNS records; to do this automatically you need access to the DNS host's API, and very few have good scoping (i.e. the API keys allows modifying everything, you can't scope the to just one record). For the *.goatcounter.com certificate (wildcards require dns-01 verification) I ... just update it manually every few months :-/ It kind of sucks, but it's the only way I could figure out without leaving the keys to the castle on the server.
I suppose that for boringproxy it's a more acceptable choice, since an expired cert usually isn't a big deal, but still...
That being said, explicit adding of domains probably isn't a bad thing.

I actually gave up on boringproxy yesterday evening as I couldn't get the ACME to work at all; the server isn't exposed directly but via Varnish (since the server also runs other things), and I couldn't really figure out a good way to fix this, so I "just used ngrok" to test that one webhook (which was my goal in the first place before I got a bit distracted by this).
I kind of hate dealing with ACME; people say it's "easy", but in my experience it's not really; the whole verification stuff it pretty error-prone, especially in situations that deviate every-so-slightly from the intended setup, and a lot of the tooling/libraries aren't that great and don't give a whole lot of feedback on failures.
I'm not sure what a good solution is; I'm not sure if there is any, as a lot of this is inherent in how ACME works. But better feedback on what exactly it's doing and what exactly is going wrong would be a good start. I mean, that error message at the start is pretty long but also contains almost no information at all. I don't know if there's a way to tell certmagic to give more information
It's also difficult for me to test this stuff, since my ISP uses carrier grade NAT and I don't have a publicly accessible IP at all (it's shared with a bunch of other subscribers), so I need to work on it by sshing to a server :-/

But people would need access to the server in order to do that, right?
I'm not sure if this is really a security issue in boringproxy as such, but rather in the ACME protocol: it assumes that if you have access to whatever a domain points at, you have rightful control over that domain. I think that's mostly a reasonable assumption. Personally I wouldn't worry too much about this, if I understand the issue correct.
The big downside of the dns-01 verification is that you need the modify the DNS records; to do this automatically you need access to the DNS host's API, and very few have good scoping (i.e. the API keys allows modifying everything, you can't scope the to just one record). For the *.goatcounter.com certificate (wildcards require dns-01 verification) I ... just update it manually every few months :-/ It kind of sucks, but it's the only way I could figure out without leaving the keys to the castle on the server.
I suppose that for boringproxy it's a more acceptable choice, since an expired cert usually isn't a big deal, but still...
That being said, explicit adding of domains probably isn't a bad thing.

I actually gave up on boringproxy yesterday evening as I couldn't get the ACME to work at all; the server isn't exposed directly but via Varnish (since the server also runs other things), and I couldn't really figure out a good way to fix this, so I "just used ngrok" to test that one webhook (which was my goal in the first place before I got a bit distracted by this).
I kind of hate dealing with ACME; people say it's "easy", but in my experience it's not really; the whole verification stuff it pretty error-prone, especially in situations that deviate every-so-slightly from the intended setup, and a lot of the tooling/libraries aren't that great and don't give a whole lot of feedback on failures.
I'm not sure what a good solution is; I'm not sure if there is any, as a lot of this is inherent in how ACME works. But better feedback on what exactly it's doing and what exactly is going wrong would be a good start. I mean, that error message at the start is pretty long but also contains almost no information at all. I don't know if there's a way to tell certmagic to give more information
It's also difficult for me to test this stuff, since my ISP uses carrier grade NAT and I don't have a publicly accessible IP at all (it's shared with a bunch of other subscribers), so I need to work on it by sshing to a server :-/

But people would need access to the server in order to do that, right?

Unfortunately no. It's not really clear from the demo, but boringproxy supports multi-tenancy. Admins can create as many users as they want, and each user can create tunnels for any domain, as long as ACME succeeds. I built it this way because I eventually want to offer boringproxy as a hosted service. The way you run GoatCounter was actually an inspiration to me. I like the idea of taking an open source project and charging people to run instances for them. I've actually been meaning to email you and ask you what lessons you've learned from this business model and whether you'd recommend it.

The big downside of the dns-01 verification is that you need the modify the DNS records

Agreed. I'm planning to avoid the DNS challenge if at all possible.

I actually gave up on boringproxy yesterday evening as I couldn't get the ACME to work at all; the server isn't exposed directly but via Varnish (since the server also runs other things), and I couldn't really figure out a good way to fix this, so I "just used ngrok" to test that one webhook (which was my goal in the first place before I got a bit distracted by this).

Sorry it didn't work out. It's definitely still beta quality. Running it behind another reverse proxy is totally untested. The intention is that for most setups, boringproxy is going to be your frontline proxy. But it's not ready for that yet either.
You're probably already aware of it since you found boringproxy, but there are more mature tunneling proxies on my list here: https://github.com/anderspitman/awesome-tunneling

I kind of hate dealing with ACME; people say it's "easy", but in my experience it's not really

TBH my overall experience with certmagic at least has been very positive. I hate dependencies, but after trying to figure out what ACME would look like to implement myself took this one on gladly. And it really has just worked. It was a bit confusing to understand some of the semantics at first because it's relatively low level, but the design is solid.

It's also difficult for me to test this stuff, since my ISP uses carrier grade NAT and I don't have a publicly accessible IP at all (it's shared with a bunch of other subscribers), so I need to work on it by sshing to a server :-/

I actually made boringproxy because I knew many people would need something like it to run the federated storage protocol I'm working on. People have big hard drives on their devices, but no way to access them easily behind NAT. I think tunneling could be a bridge to a true p2p internet. Once tunneling is robust and easy to do, developers can build p2p apps on top of it. Once the apps are there, hopefully there will be enough incentive to switch to ipv6 and drop NAT.

But people would need access to the server in order to do that, right?

Unfortunately no. It's not really clear from the demo, but boringproxy supports multi-tenancy. Admins can create as many users as they want, and each user can create tunnels for any domain, as long as ACME succeeds. I built it this way because I eventually want to offer boringproxy as a hosted service. The way you run GoatCounter was actually an inspiration to me. I like the idea of taking an open source project and charging people to run instances for them. I've actually been meaning to email you and ask you what lessons you've learned from this business model and whether you'd recommend it.

The big downside of the dns-01 verification is that you need the modify the DNS records

Agreed. I'm planning to avoid the DNS challenge if at all possible.

I actually gave up on boringproxy yesterday evening as I couldn't get the ACME to work at all; the server isn't exposed directly but via Varnish (since the server also runs other things), and I couldn't really figure out a good way to fix this, so I "just used ngrok" to test that one webhook (which was my goal in the first place before I got a bit distracted by this).

Sorry it didn't work out. It's definitely still beta quality. Running it behind another reverse proxy is totally untested. The intention is that for most setups, boringproxy is going to be your frontline proxy. But it's not ready for that yet either.
You're probably already aware of it since you found boringproxy, but there are more mature tunneling proxies on my list here: https://github.com/anderspitman/awesome-tunneling

I kind of hate dealing with ACME; people say it's "easy", but in my experience it's not really

TBH my overall experience with certmagic at least has been very positive. I hate dependencies, but after trying to figure out what ACME would look like to implement myself took this one on gladly. And it really has just worked. It was a bit confusing to understand some of the semantics at first because it's relatively low level, but the design is solid.

It's also difficult for me to test this stuff, since my ISP uses carrier grade NAT and I don't have a publicly accessible IP at all (it's shared with a bunch of other subscribers), so I need to work on it by sshing to a server :-/

I actually made boringproxy because I knew many people would need something like it to run the federated storage protocol I'm working on. People have big hard drives on their devices, but no way to access them easily behind NAT. I think tunneling could be a bridge to a true p2p internet. Once tunneling is robust and easy to do, developers can build p2p apps on top of it. Once the apps are there, hopefully there will be enough incentive to switch to ipv6 and drop NAT.

Oh and for a lot of ngrok-like tasks (webhooks being a great example), you should check out my https://patchbay.pub service. Even if you don't use it I think you'll like it :)

Oh and for a lot of ngrok-like tasks (webhooks being a great example), you should check out my https://patchbay.pub service. Even if you don't use it I think you'll like it :)

Ah right, I never got beyond trying to trying to start the server, so I never managed to see the web UI 😅

 Sorry it didn't work out. It's definitely still beta quality. Running it behind another reverse proxy is totally untested. The intention is that for most setups, boringproxy is going to be your frontline proxy. But it's not ready for that yet either.

Well, it's not really your fault – it's just that ACME in general is pretty painful in these kind of scenarios. It took me quite a while to get goatcounter to work well with it too.
Anyway, I tried running it without proxy too (on a different port), but couldn't get that to work either; I'm not sure if the tls-alpn-01 challenge supports that or if it always tries to use port 80/443?

Ah right, I never got beyond trying to trying to start the server, so I never managed to see the web UI 😅

 Sorry it didn't work out. It's definitely still beta quality. Running it behind another reverse proxy is totally untested. The intention is that for most setups, boringproxy is going to be your frontline proxy. But it's not ready for that yet either.

Well, it's not really your fault – it's just that ACME in general is pretty painful in these kind of scenarios. It took me quite a while to get goatcounter to work well with it too.
Anyway, I tried running it without proxy too (on a different port), but couldn't get that to work either; I'm not sure if the tls-alpn-01 challenge supports that or if it always tries to use port 80/443?

Yeah I'm pretty sure ACME (or maybe Let's Encrypt) specifies you must use ports 80/443.

Yeah I'm pretty sure ACME (or maybe Let's Encrypt) specifies you must use ports 80/443.

At times it may be sufficient to switch to the Let's Encrypt staging ACME endpoint for tunnels whose configuration is not yet stabilised, i.e. switching through different Client, Server or Passthrough conditions and hitting the rate limit, also keeping #42 in mind.
A simple DNS check prior requesting a certificate is also what Kubernetes' cert-manager does.

Yeah adding support for staging ACME would definitely be a good idea. It's very annoying when you have to wait a day to reuse a specific address. I'll add an issue.

The bespoken issue can be found at #51, in case s/o wonders.


XDebug fails to connect because vscode-php-debug is listening on IPv6

PHP version:  7.1.20
XDebug version:  2.6.0
Adapter version: 1.12.3
Your launch.json:
{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Listen for XDebug",
      "type": "php",
      "request": "launch",
      "pathMappings": {
        "/portal": "${workspaceFolder}",
      },
      "port": 9000,
      "log": true
    },
    {
      "name": "Launch currently open script",
      "type": "php",
      "request": "launch",
      "program": "${file}",
      "cwd": "${fileDirname}",
      "port": 9000
    }
  ]
}

XDebug php.ini config:
zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20160303/xdebug.so
xdebug.remote_enable=1
xdebug.remote_autostart=1
xdebug.remote_connect_back=1
xdebug.remote_log=/var/log/xdebug.log
xdebug.remote_host=host.docker.internal

XDebug logfile (from setting xdebug.remote_log in php.ini):
Log opened at 2018-08-01 02:13:59
I: Checking remote connect back address.
I: Checking header 'HTTP_X_FORWARDED_FOR'.
I: Checking header 'REMOTE_ADDR'.
I: Remote address found, connecting to <IP_ADDRESS>:9000.
W: Creating socket for '<IP_ADDRESS>:9000', poll success, but error: Operation now in progress (29).
E: Could not connect to client. :-(
Log closed at 2018-08-01 02:13:59

Adapter logfile (from setting "log": true in launch.json):
<- launchResponse
Response {
  seq: 0,
  type: 'response',
  request_seq: 2,
  command: 'launch',
  success: true }

Code snippet to reproduce:
<?php
phpinfo();

Also relevant:
$ lsof -nP -i4TCP:9000
COMMAND     PID            USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
Code\x20H 46903 matthewscharley   28u  IPv6 0x32d841ebd93c8e67      0t0  TCP *:9000 (LISTEN)

It would be great to be able to specify a bind address, which would then let me solve this easily. Alternatively, binding to both IPv4 and IPv6 would be an even better solution.

Also relevant, the top of my hosts file:
$ head /etc/hosts
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
<IP_ADDRESS>       localhost
<IP_ADDRESS> broadcasthost
::1             localhost

That should mean that the IPv4 address should be preferred, but it's not working for this project for some reason.

I've seen #162 and according to the docs the advice given there is wrong: https://nodejs.org/api/net.html#net_server_listen_port_host_backlog_callback
If host is omitted, the server will accept connections on the unspecified IPv6 address (::) when IPv6 is available, or the unspecified IPv4 address (<IP_ADDRESS>) otherwise.

This means that it's actually impossible to specifically setup an IPv4 address to listen on currently. Again, allowing a bind address would solve this problem. Not sure how hard this would be to add but looks like it shouldn't be too hard from the code - just need to add a hostname here from an argument, the same as the port currently is.
https://github.com/felixfbecker/vscode-php-debug/blob/master/src/phpDebug.ts#L313

+1

Any progress on this issue?

The PR I put up still merges cleanly and worked for me when I was dealing with this issue, so this is with @felixfbecker

For what it's worth, I ended up finding out this wasn't the actual issue on my Mac OS host after all, but it's been sufficiently long that I forget what the actual issue was now. This PR did work as intended though so I left it up.


One Form Action

i want to place my avatar upload file field in the same form which has user's email and other fields and do the same with one controller action is there any ways to do this? for example form should look like this?
<%= form_for(@user, @uploader) do |f| %>
<%= f.hidden_field :key %>
<%= f.text_field :email:  %>
<%= f.file_field :avatar:  %>

    <%= f.submit %>

<% end %>

This does not work because the form can only be submitted to your server of S3, not both.

your server OR s3, not both


Install instruction improvements

Your 'Installation for R Users' says to start with  > library(devtools)
but that triggers an error to first download and install Rtools (can't be done from the packages menu).  I suspect that is just the start of a dependency cascade of other libraries to be installed.

Please have someone with a fresh install of R test the instructions, someone who is not a developer and may be missing some components that are taken for granted, and then revise your pre-installation requirements.
There is also no comment about whether someone with R can also install the stand-alone version without causing problems.
Thanks.

The stand alone version will not interfere with the R version, and vice versa.
The installation instructions have been updated.
Thanks for the feedback!

Joe, Thanks.
I will check it out tomorrow or Monday.
-=Steve
Steve Tessler, PhD
Ecologist / Data Manager
<EMAIL_ADDRESS>USGS Lawrenceville, NJ
Work Cell (preferred)<PHONE_NUMBER>, NJ Office<PHONE_NUMBER>
USGS - NJ Water Science Center http://nj.usgs.gov/
On Thu, Sep 3, 2015 at 2:56 PM, T. Joe Mills<EMAIL_ADDRESS>wrote:

The stand alone version will not interfere with the R version, and vice
versa.
The installation instructions have been updated.
Thanks for the feedback!
—
Reply to this email directly or view it on GitHub
https://github.com/USGS-R/WQ-Review/issues/17#issuecomment-137541948.


Lines starting with # should be a valid comment line

Many languages use # to begin a comment line - can this be added as a valid comment line?

Extension updated to support # comments. Please see if it works for you now.


basestring does not exist in Python 3. use str instead.

On Sun, Mar 15, 2015 at 03:54:21PM -0700, Nick Young wrote:

basestring does not exist in Python 3. use str instead.

I think this is a duplicate of #220.  See that issue for notes about
where check.py comes from and why I don't think we want amy-side
tweaks.

I'd agree this should be refactored out into an external package on pypi. Should I take care of that and then submit a pull request with the change to requirements.txt? For reference, the view that this is currently breaking is validate_event.

On Sun, Mar 15, 2015 at 06:34:53PM -0700, Nick Young wrote:

I'd agree this should be refactored out into an external package on
pypi. Should I take care of that and then submit a pull request with
the change to requirements.txt? For reference, the view that this is
currently breaking is validate_event.

I think the first step is fixing the script in it's upstream
swcarpentry/workshop-template location to work with both Python 2 and
3.  We should all agree that that's a good thing ;).
Personally, I'd also support anyone who wanted to split the script
into it's own package and push it to PyPI, but I'm not sure where the
swcarpentry/workshop-template maintainers stand on that issue.  It
can't hurt to ask them though :).

Upstream is already Python 2/3 compatible, it's just that amy's copy of check.py is outdated

On Sun, Mar 15, 2015 at 08:57:56PM -0700, Nick Young wrote:

Upstream is already Python 2/3 compatible, it's just that amy's copy
of check.py is outdated

So then go the PyPI route if the swcarpentry/workshop-template folks
sign off on it.  Otherwise, just copy the new version from
swcarpentry/workshop-template into this repository?  Maybe that's what
you are doing?  For bonus points on the copy, cherry-pick the commits
from swcarpentry/workshop-template to preserve authorship information
and the motivation behind the changes.

@gvwilson thoughts?

Please just copy in the new version for now - we need to tackle our repository proliferation + packaging problem soon, but we're not there yet.

Closing and reopening at #232


Jraj/demo lab

seems like this is so old that it can't be pertinent anymore.  If you still want to merge it, please submit a new pull request.


ARGUMENT_CAPTURING_OFF should not show up

ARGUMENT_CAPTURING_OFF should not show up but does, a lot.

This reproduces the issue
java "-javaagent:./javaAgent/build/libs/javaAgent.jar=log:INFO,out:/tmp,filter:nameMatches(.TestFilters[AB].)" -jar ./javaExampleApp/build/libs/javaExampleApp.jar

ARGUMENT_CAPTURING_OFF does not exist anymore


cowboy_clock crashes during lookup of rfc1123

I get below error log intermittently, possibly during high load in the system.
2017-10-20 04:56:50.535 [error] <0.720.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{case_clause,[http,<0.4193.0>,1,<0.4194.0>,badarg,[{ets,lookup_element,[cowboy_clock,rfc1123,2],[]},{cowboy_clock,rfc1123,0,[{file,"/drone/src/code.xaptum.xyz/xaptum-development/xmx/_build/default/lib/cowboy/src/cowboy_clock.erl"},{line,54}]},{cowboy_req,response_headers,2,[{file,"/drone/src/code.xaptum.xyz/xaptum-development/xmx/_build/default/lib/cowboy/src/cowboy_req.erl"},{line,778}]},{cowboy_req,do_reply,4,[{file,"/drone/src/code.xaptum.xyz/xaptum-development/xmx/_build/defau..."},...]},...]]},...}}
2017-10-20 04:56:50.588 [error] <0.4197.0>@cowboy_clock:rfc1123:54 CRASH REPORT Process <0.4197.0> with 0 neighbours crashed with reason: bad argument in call to ets:lookup_element(cowboy_clock, rfc1123, 2) in cowboy_clock:rfc1123/0 line 54
2017-10-20 04:56:50.588 [error] <0.720.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{case_clause,[http,<0.4196.0>,1,<0.4197.0>,badarg,[{ets,lookup_element,[cowboy_clock,rfc1123,2],[]},{cowboy_clock,rfc1123,0,[{file,"/drone/src/code.xaptum.xyz/xaptum-development/xmx/_build/default/lib/cowboy/src/cowboy_clock.erl"},{line,54}]},{cowboy_req,response_headers,2,[{file,"/drone/src/code.xaptum.xyz/xaptum-development/xmx/_build/default/lib/cowboy/src/cowboy_req.erl"},{line,778}]},{cowboy_req,do_reply,4,[{file,"/drone/src/code.xaptum.xyz/xaptum-development/xmx/_build/defau..."},...]},...]]},...}}```

Is there anything else? Because I think this could happen for one of two cases:

the cowboy_clock process has crashed and is being restarted; the ets table is temporarily gone
there is a bug in ets and the writes are not atomic

Would be good if you can confirm whether cowboy_clock restarted (I guess the pid number can give a good hint) and if it has, see if you can find the crash reason.

Hi Irina,
I have the same issue when I did not start cowboy as application. The cowboy_clock process is started as worker by cowboy_sup that is started by cowboy_app.
I hope it will help in your debugging.
Regards,
Alexei.

Hi Alexei,
Thanks so much!
I have cowboy app running for sure as it's working for me all along.
However, I saw this error again and this time I was mindful of what you
said above and realized that this happened while my server was being
restarted (obviously along with all the other apps restarting with it -- so
looks like something is still trying to use cowboy when it's already
stopped/stopping and that's why I get this error).   I can't be sure now
about the other times I was seeing it, maybe it was always this situation.
I'll keep an eye on it to see if I can reproduce when the server isn't
restarting, but I suspect now that this was probably always the case.
Cheers,
Irina.
On Sun, Oct 22, 2017 at 11:08 PM, alekras<EMAIL_ADDRESS>wrote:

Hi Irina,
I have the same issue when I did not start cowboy as application. The
cowboy_clock process is started as worker by cowboy_sup that is started by
cowboy_app.
I hope it will help in your debugging.
Regards,
Alexei.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ninenines/cowboy/issues/1228#issuecomment-338542884,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALcepvOrNw0OFyi69vsi0D-kk2XtcWABks5svBFFgaJpZM4QA15J
.

I can see that happening if Ranch supervises the connections because those will still run while Cowboy is being restarted, and if Cowboy is not there then cowboy_clock isn't either, so the ets table doesn't exist (anymore).
I suppose a graceful way to deal with that is to catch the error and then compute the value directly if the ets table is gone.

Hi Loïc,
I've been watching our logs and seems like it does happen upon server exit.
So I guess it isn't really an issue.
Thanks so much for your help,
Irina.
On Mon, Oct 23, 2017 at 11:01 AM, Loïc Hoguin<EMAIL_ADDRESS>wrote:

I can see that happening if Ranch supervises the connections because those
will still run while Cowboy is being restarted, and if Cowboy is not there
then cowboy_clock isn't either, so the ets table doesn't exist (anymore).
I suppose a graceful way to deal with that is to catch the error and then
compute the value directly if the ets table is gone.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ninenines/cowboy/issues/1228#issuecomment-338708338,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALcepieu9wPpm-5c-0AXFCSEfrsYKDcmks5svLhSgaJpZM4QA15J
.

Cheers, good to know! I will still do the proposed change in my previous comment so that these logs don't appear anymore and to not have connections get killed because of this.

Fixed locally, this will be part of 2.1. Thanks!

Awesome!  Thanks so much Loïc!


Add comment icon

@geekrainy what is gentie? Can u give basic link and some description on it?

@ivan-nginx https://gentie.163.com/
It's a comment plugin. : )

From this pull Next support 4 comments system. :+1:

Duoshuo
Disqus
Hypercomments
Gentie

Thanks. :+1:

@geekrainy there is Netease code in #1543. Is that need thing?

@ivan-nginx #1543 It provides the same code. Do not need merge.


Memory Leak Causes Crash

Valid command!
 bkchr wants to tip nazar-pc (1vSxzbyz2cJREAuVWjhXUT1ds8vBzoxn2w4asNpusQKwjJd on polkadot) a medium tip for pull request https://github.com/paritytech/substrate/pull/10388.

<--- Last few GCs --->

[28796:0x6829f40]<PHONE_NUMBER> ms: Mark-sweep 480.4 (502.6) -> 476.1 (502.4) MB, 929.6 / 0.0 ms  (average mu = 0.127, current mu = 0.017) allocation failure scavenge might not succeed
[28796:0x6829f40]<PHONE_NUMBER> ms: Mark-sweep 480.3 (502.4) -> 476.2 (502.6) MB, 938.5 / 0.0 ms  (average mu = 0.075, current mu = 0.016) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xafd010 node::Abort() [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 2: 0xa141fb node::FatalError(char const*, char const*) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 3: 0xce746e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 4: 0xce77e7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 5: 0xeb5c85  [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 6: 0xeb6766  [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 7: 0xec471e  [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 8: 0xec5140 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags)
[/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 9: 0xec7fae v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
10: 0xe899da v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
11: 0x11fea76 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
12: 0x15e7fb9  [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
Aborted
[nodemon] app crashed - waiting for file changes before starting...

Another:
2022-04-05 08:08:24        REGISTRY: Unknown signed extensions CheckNonZeroSender found, treating them as no-effect
2022-04-05 08:08:24        API/INIT: Runtime version updated to spec=9180, tx=12
2022-04-05 08:08:24        REGISTRY: Unknown signed extensions CheckNonZeroSender found, treating them as no-effect
2022-04-05 08:08:25        API/INIT: Runtime version updated to spec=9180, tx=12
2022-04-05 08:08:25        REGISTRY: Unknown signed extensions CheckNonZeroSender found, treating them as no-effect
2022-04-05 08:08:27        API/INIT: Runtime version updated to spec=9180, tx=12
2022-04-05 08:08:27        REGISTRY: Unknown signed extensions CheckNonZeroSender found, treating them as no-effect

<--- Last few GCs --->

[30309:0x624ef40]<PHONE_NUMBER> ms: Mark-sweep 491.8 (503.2) -> 490.9 (505.2) MB, 1016.9 / 0.0 ms  (average mu = 0.707, current mu = 0.008) allocation
failure scavenge might not succeed
[30309:0x624ef40]<PHONE_NUMBER> ms: Mark-sweep (reduce) 493.0 (505.2) -> 491.6 (502.7) MB, 732.8 / 0.0 ms  (+ 150.6 ms in 30 steps since start of marking, biggest step 6.5 ms, walltime since start of marking 908 ms) (average mu = 0.552, current mu = 0.028) a

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xafd010 node::Abort() [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 2: 0xa141fb node::FatalError(char const*, char const*) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 3: 0xce746e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 4: 0xce77e7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 5: 0xeb5c85  [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 6: 0xec5401 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 7: 0xec7fae v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 8: 0xe899da v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
 9: 0x11fea76 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
10: 0x15e7fb9  [/home/shawntabrizi/.nvm/versions/node/v16.10.0/bin/node]
Aborted
[nodemon] app crashed - waiting for file changes before starting...

Can't find in logs, could be outdated


gitFull: add patch to workaround gitk issue in Darwin

Description of changes

Without this patch gitk just show an empty window.
Before:

After:

Not sure the impact in other platforms, and gitk works correctly for me in Linux at least. So applying this patch only for Darwin.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

CC @NixOS/darwin-maintainers.

For context: this patch is already used by Homebrew for more than 2 years: https://github.com/Homebrew/homebrew-core/commit/53d27653341208e7cf90485b2227924847ae648e.

Result of nixpkgs-review pr 325645 run on aarch64-darwin 1

2 packages built:

git-doc (git-doc.doc ,gitFull.doc)
gitFull


Engine doesn't serialize lists of dicts of params

Passing in a list of dicts to a param sweep doesn't serialize to the v2 protos
params=[{'a': 1, 'b': 2}, {'a':2, 'b':3}]

Being solved by PR #2080

You should now be able to serialize flattened sweeps and lists of dicts, as long as you make them a ListSweep.
e.g.  params=ListSweep([{'a': 1, 'b': 2}, {'a':2, 'b':3}])


[FLINK-24352] [flink-table-planner] Add null check for temporal table check on SqlSnapshot

What is the purpose of the change
Purpose of this change is to resolve https://issues.apache.org/jira/browse/FLINK-24352 and it's duplicate https://issues.apache.org/jira/browse/FLINK-24956.
The below test which uses a lookup table join with a CTE fails due to a NullPointerException
package foo.bar

import org.apache.flink.api.scala.createTypeInformation
import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
import org.apache.flink.table.api.DataTypes
import org.apache.flink.table.api.Schema
import org.apache.flink.table.api.bridge.scala.StreamTableEnvironment
object Test {
  final case class Person(name: String, age: Int)

  def main(args: Array[String]): Unit = {
    val ee = StreamExecutionEnvironment.getExecutionEnvironment
    val te = StreamTableEnvironment.create(ee)
    val personSchema = Schema.newBuilder().column("name", DataTypes.STRING()).column("age", DataTypes.INT()).build()
    val x = ee.fromCollection(List(Person("a", 1)))
    te.createTemporaryView(
      "my_table",
      x,
      personSchema
    )
    val y = ee.fromCollection(List(Person("b", 2)))
    te.createTemporaryView(
      "my_table_2",
      y,
      personSchema
    )
    val res =
      te.executeSql("""
                      |WITH A AS (
                      |  select name, age + 1 from my_table
                      |),
                      |B AS (
                      |  select name, age + 2 from my_table_2
                      |)
                      |
                      |SELECT A.name, B.age
                      |FROM A
                      |JOIN B
                      |FOR SYSTEM_TIME AS OF PROCTIME() on (A.name = B.name)
                      |""".stripMargin)
    res.print()
  }
}

@tsreaper waiting on your comments. I'm not entirely sure a null check is sufficient to determine the underlying table is actually temporal.
Brief change log

Add null check for SqlValidatorImpl.validateSnapshot before checking the underling table, which may be null in the case the expression is a CTE.

Verifying this change
This change added tests and can be verified as follows:

Added a test to TableEnvironmentITCase which validates calling a lookup join on a CTE no longer throws a NPE.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@tsreaper I have fixed the PR name and made the check more specific. Additionally I replaced the test ITCase test with a test in LookupJoinTest as you mentioned.

cc. @godfreyhe do you have any other thoughts about this solution?

cc. @godfreyhe do you have any other thoughts about this solution?

@godfreyhe Would appreciate any pointers.

@YuvalItzchakov Thanks for the contribution, I'm be fine with the current solution even it not very well, since we do not check whether the query in with is temporal (just like the following branch you added). That do not affect the result because we will check it again in planner rules.

Should we create a follow-up ticket for the issue you've mentioned @godfreyhe ?

@godfreyhe

since we do not check whether the query in with is temporal

I don't quite understand this. I think with is always temporal?

@godfreyhe @tsreaper I have fixed the PR comments. Are we good with merging this?

@godfreyhe Would appreciate your reply so we can merge this into 1.14.1 as well

Anyone?

Let me see if I can find someone who can help

hi, sorry for jumping in, but I think current fix didn't cover the other case that the getTable can be null. If there are sometime the getTable is null but can not be ignored in validation, we should throw an helpful exception instead of NPE.

hi, sorry for jumping in, but I think current fix didn't cover the other case that the getTable can be null. If there are sometime the getTable is null but can not be ignored in validation, we should throw an helpful exception instead of NPE.

@wenlong88 I don't get this. This PR is meant to fix the case when getTable() returns null. Could you elaborate more on this and preferably give us an counter-example?

@tsreaper @godfreyhe @MartijnVisser Guys this has been waiting for a month now although the PR is ready.
This fix is affecting the ability of my consumers to use Flink and I'd really like to get this merged.
As there is no more critical issues re table nullability, is there anything else stalling us?

@YuvalItzchakov I add assert ns.getTable() != null  before calling ns.getTable().isTemporal() and merge the pr. Thanks for the contribution


Legacy 2 MVC

What are the plans in the translation of legacy code to MVC ? Do you need help ?

We are still working in some core areas (mainly firewall), but we can't manage too many changes at once.
Help is certainly welcome, preferable for features which aren't too big and can easily be migrated, such as diagnostics pages (dns lookup, ping, ...).
Every pull request costs us time to manage and test, which is fine, but we need to spend the limited time we have wisely.
https://github.com/opnsense/core/issues/1774 for example also needs work, we did pull in the code, but can't spend enough time at the moment to finish it.
Thanks for your support!

@kekek2 how about certificate Code .. would close #1905

@mimugmail @kekek2  I would suggest starting with smaller things, the trust/certificate section is quite a blob of work, which might result in quite some work for the core team to review as well (which we might not have available at the moment).

The trust/certificates are almost ready, tested. Will soon make pull-request.


POC for 1D images (e.g. line plots)

Description
Following up on some of the discussion around 1D images from last weeks dev meeting, I put together this very rough and ready PR that does the bare minimum to support 1D plots for the image layer.
The idea is that we'd now allow viewer.dims.ndisplay=1 and we'd then do 1D slicing on all our layers and 1D rendering. For now I just hacked it together so that the image layer works. It also ignores a lot of additional possible settings, but it is pretty cool.
We can discuss whether we like this direction or not, but I'm already planning to make use of this branch with some 1D wave equations simulations I've been playing with where I have a time axis and a 1D plot (so a 2D dataset) but I want to look at 1D plots of it. I also want to use napari-animation to make movies of the wave propagation, so having this include in napari instead of something like matplotlib is very helpful.
Here's a movie of 1D slicing through a 2D image of a cell:
viewer.open_sample(plugin='scikit-image', sample='cell')
viewer.dims.ndisplay = 1

https://user-images.githubusercontent.com/6531703/119244514-5ab69a80-bb26-11eb-99d1-8c60fb792bb4.mov
Curious what others think cc @tlambert03 @jni @andy-sweet
Type of change

[ ] Bug-fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

References

How has this been tested?

[ ] example: the test suite for my feature covers cases x, y, and z
[ ] example: all tests pass with my change

Final checklist:

[ ] My PR is the minimum possible work for the desired functionality
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] If I included new strings, I have used trans. to make them localizable.
For more information see our translations guide.

We can discuss whether we like this direction or not, but I'm already planning to make use of this branch with some 1D wave equations simulations I've been playing with where I have a time axis and a 1D plot (so a 2D dataset) but I want to look at 1D plots of it. I also want to use napari-animation to make movies of the wave propagation, so having this include in napari instead of something like matplotlib is very helpful.

This looks like a lot of fun. Can I select the axis of the slice? X vs Y ? (a custom axis eventually?)

This looks like a lot of fun. Can I select the axis of the slice? X vs Y ? (a custom axis eventually?)

Yup, hitting the transpose / roll axis buttons now will change what get's sliced, though the sliced plot is always shown horizontally.

I definitely like the concept of ndisplay = 1 ... but not quite sure that the main canvas is the right spot for it.  Seems very specific to image/labels, in which case a linescan kind of feels appropriate flexible.
(if the motivation for this POC is fundamentally different than a linescan, let me know!)

(if the motivation for this POC is fundamentally different than a linescan, let me know!)

I was actually imaging this as being more generally useful, for example this movie of watching a 1D wave propagate through a medium of variable speed was the reason I made the branch. Note the data here is 2D ('t', 'x') and even though playing the movie is a special case of a linescan for an application like this it is nice to have the plot be inside the main canvas.
I can understand if people feel like 1D wave simulation is a bit unique of an application, that isn't so directly tied to imaging, but I do like this functionality a lot for this purpose!! We can definitely let this sit for a while before deciding what to do next, I'm happy using my branch right now.
https://user-images.githubusercontent.com/6531703/119280929-6a96b300-bbe8-11eb-9c03-aeed0329f185.mov

This was discussed with the group the other day and there was some support for the use case from @jni, but this PR still needs some work before it is ready to merge. In particular we need to think about other layer types. We might need to watch out for some corner cases too around passing 1D arrays. I have created #3562 to track this idea and will close this now. Please weigh in there if you are interested in this functionality


FIX: remove unused_import, this caused fe compile failed

remove unused_import, this caused fe compile failed

Hi @stalary , thank u for your PR, but this bug has been fixed in #3826.
So I will close this PR.


Material system GLSL compile error on Intel

Tested with a clean homepath, only setting the 2 cvars to enable material system.
Calling GetRefAPI…
GL_VENDOR: Intel
GL_RENDERER: Intel(R) UHD Graphics 630
GL_VERSION: 3.2.0 - Build <IP_ADDRESS>5
GL_MAX_TEXTURE_SIZE: 16384
GL_SHADING_LANGUAGE_VERSION: 1.50 - Build <IP_ADDRESS>5
GL_MAX_VERTEX_UNIFORM_COMPONENTS 4096
GL_MAX_VERTEX_ATTRIBS 16
Occlusion query bits: 64
GL_MAX_DRAW_BUFFERS: 8
GL_TEXTURE_MAX_ANISOTROPY_EXT: 16.000000
GL_MAX_RENDERBUFFER_SIZE: 16384
GL_MAX_COLOR_ATTACHMENTS: 8
PIXELFORMAT: color(16-bits)
MODE: -2, 1920 x 1080 fullscreen hz: N/A
Using OpenGL version 3.2, requested: 3.2
^x0f0Using OpenGL 3.x context.
^x0f0Using an OpenGL core profile.
OpenGL 3.x context is not forward compatible.
^x0f0Using OpenGL extensions: GL_ARB_half_float_pixel GL_ARB_texture_float GL_EXT_gpu_shader4 GL_ARB_gpu_shader5 GL_EXT_texture_integer GL_ARB_texture_rg GL_ARB_texture_gather GL_ARB_provoking_vertex GL_EXT_texture_compression_s3tc GL_ARB_texture_compression_rgtc GL_EXT_texture_filter_anisotropic GL_ARB_half_float_vertex GL_ARB_framebuffer_object GL_ARB_get_program_binary GL_ARB_buffer_storage GL_ARB_uniform_buffer_object GL_ARB_map_buffer_range GL_ARB_sync GL_ARB_depth_clamp GL_ARB_compute_shader GL_ARB_bindless_texture GL_ARB_shader_draw_parameters GL_ARB_shader_storage_buffer_object GL_ARB_multi_draw_indirect GL_ARB_shading_language_420pack GL_ARB_explicit_uniform_location GL_ARB_shader_image_load_store GL_ARB_shader_atomic_counters GL_ARB_indirect_parameters
^x0f0Using S3TC (DXTC) texture compression.
^x0f0Using GPU vertex skinning with max 233 bones in a single pass, models are hardware accelerated.
Extracting VM module cgame-amd64.nexe from c:\users\meh\Documents\My Games\Unvanquished\pkg/unvanquished_0.54.0.dpk...
Loading VM module cgame-amd64.nexe...
Using loader args:  c:\unv\Unvanquished\build\vs-deps10\Release/nacl_loader.exe -v -B c:\unv\Unvanquished\build\vs-deps10\Release/irt_core-amd64.nexe -e -i 100:1868 -- c:\unv\tmp\hom323/cgame-amd64.nexe 100
Loaded VM module in 583 msec
Loading Lua plugin using a new Lua state.
Loaded font face 'unifont' [regular] from 'fonts/unifont.ttf'.
Loaded font face 'Roboto' [bold] from 'fonts/Roboto-Bold.ttf'.
Loaded font face 'Roboto' [italic, bold] from 'fonts/Roboto-BoldItalic.ttf'.
Loaded font face 'Roboto' [italic] from 'fonts/Roboto-Italic.ttf'.
Loaded font face 'Roboto' [regular] from 'fonts/Roboto-Regular.ttf'.
Loaded font face 'FontAwesome' [regular] from 'fonts/fontawesome-webfont.ttf'.
Loaded font face 'DejaVu Sans' [regular] from 'fonts/DejaVuSansCondensed.ttf'.
Loaded font face 'Material Icons' [regular] from 'fonts/MaterialIcons-Regular.ttf'.
Invalidating shader binary cache
^3Warn: Source for shader program genericMaterial:
   0: #version 150 core
   1: #define HAVE_EXT_gpu_shader4 1
   2: #extension GL_ARB_gpu_shader5 : require
   3: #define HAVE_ARB_gpu_shader5 1
   4: #extension GL_ARB_texture_gather : require
   5: #define HAVE_ARB_texture_gather 1
   6: #define HAVE_EXT_texture_integer 1
   7: #define HAVE_ARB_texture_rg 1
   8: #define HAVE_ARB_uniform_buffer_object 1
   9: #extension GL_ARB_bindless_texture : require
  10: #define HAVE_ARB_bindless_texture 1
  11: #extension GL_ARB_shader_draw_parameters : require
  12: #define HAVE_ARB_shader_draw_parameters 1
  13: #extension GL_ARB_shader_storage_buffer_object : require
  14: #define HAVE_ARB_shader_storage_buffer_object 1
  15: #define IN in
  16: #define OUT(mode) mode out
  17: #define textureCube texture
  18: #define texture2D texture
  19: #define texture2DProj textureProj
  20: #define texture3D texture
  21: OUT(flat) int in_drawID;
  22: OUT(flat) int in_baseInstance;
  23: #define drawID gl_DrawIDARB
  24: #define baseInstance gl_BaseInstanceARB
  25:
  26: #ifndef r_highPrecisionRendering
  27: #define r_highPrecisionRendering 1
  28: #endif
  29: #ifndef r_dynamicLight
  30: #define r_dynamicLight 1
  31: #endif
  32: #ifndef r_dynamicLightRenderer
  33: #define r_dynamicLightRenderer 1
  34: #endif
  35: #ifndef r_precomputedLighting
  36: #define r_precomputedLighting 1
  37: #endif
  38: #ifndef r_vertexSkinning
  39: #define r_vertexSkinning 1
  40: #endif
  41: const int MAX_GLSL_BONES = 233;
  42: #ifndef r_halfLambertLighting
  43: #define r_halfLambertLighting 1
  44: #endif
  45: #ifndef r_normalMapping
  46: #define r_normalMapping 1
  47: #endif
  48: #ifndef r_specularMapping
  49: #define r_specularMapping 1
  50: #endif
  51: #ifndef r_physicalMapping
  52: #define r_physicalMapping 1
  53: #endif
  54: #ifndef r_glowMapping
  55: #define r_glowMapping 1
  56: #endif
  57: #ifndef r_zNear
  58: #define r_zNear 3.00000000e+00
  59: #endif
  60: #define USE_MATERIAL_SYSTEM
  61:
  62: struct Material {
  63:   mat4 u_TextureMatrix;
  64:   vec4 u_ColorModulate;
  65:   vec4 u_Color;
  66:   uvec2 u_ColorMap;
  67:   uvec2 u_DepthMap;
  68:   float u_AlphaThreshold;
  69:   float u_InverseLightFactor;
  70:   float u_VertexInterpolation;
  71:   float u_DepthScale;
  72:   float u_Time;
  73:   int material_padding0;
  74:   int material_padding1;
  75:   int material_padding2;
  76: };
  77:
  78: layout(std430, binding = 0) readonly buffer materialsSSBO {
  79:   Material materials[];
  80: };
  81:
  82: #define u_TextureMatrix materials[baseInstance].u_TextureMatrix
  83: #define u_ColorModulate materials[baseInstance].u_ColorModulate
  84: #define u_Color materials[baseInstance].u_Color
  85: #define u_ColorMap_initial uvec2( materials[baseInstance].u_ColorMap )
  86: #define u_DepthMap_initial uvec2( materials[baseInstance].u_DepthMap )
  87: #define u_AlphaThreshold materials[baseInstance].u_AlphaThreshold
  88: #define u_InverseLightFactor materials[baseInstance].u_InverseLightFactor
  89: #define u_VertexInterpolation materials[baseInstance].u_VertexInterpolation
  90: #define u_DepthScale materials[baseInstance].u_DepthScale
  91: #define u_Time materials[baseInstance].u_Time
  92:
  93: uniform vec3 u_ViewOrigin;
  94: uniform vec3 u_ViewUp;
  95: uniform mat4 u_ModelMatrix;
  96: uniform mat4 u_ModelViewProjectionMatrix;
  97:
  98: #ifndef r_AmbientScale
  99: #define r_AmbientScale 1.00000000e+00
 100: #endif
 101: #ifndef r_SpecularScale
 102: #define r_SpecularScale 1.00000000e+00
 103: #endif
 104: #ifndef r_zNear
 105: #define r_zNear 3.00000000e+00
 106: #endif
 107: #ifndef M_PI
 108: #define M_PI 3.14159274e+00
 109: #endif
 110: #ifndef MAX_SHADOWMAPS
 111: #define MAX_SHADOWMAPS 5
 112: #endif
 113: #ifndef MAX_REF_LIGHTS
 114: #define MAX_REF_LIGHTS 1024
 115: #endif
 116: #ifndef TILE_SIZE
 117: #define TILE_SIZE 16
 118: #endif
 119: #ifndef r_FBufSize
 120: #define r_FBufSize vec2(1.92000000e+03, 1.08000000e+03)
 121: #endif
 122: #ifndef r_tileStep
 123: #define r_tileStep vec2(8.33333377e-03, 1.48148146e-02)
 124: #endif
 125: /*
 126: ===========================================================================
 127: Copyright (C) 2006-2011 Robert Beckebans<EMAIL_ADDRESS> 128:
 129: This file is part of XreaL source code.
 130:
 131: XreaL source code is free software; you can redistribute it
 132: and/or modify it under the terms of the GNU General Public License as
 133: published by the Free Software Foundation; either version 2 of the License,
 134: or (at your option) any later version.
 135:
 136: XreaL source code is distributed in the hope that it will be
 137: useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
 138: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 139: GNU General Public License for more details.
 140:
 141: You should have received a copy of the GNU General Public License
 142: along with XreaL source code; if not, write to the Free Software
 143: Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 144: ===========================================================================
 145: */
 146:
 147: /* generic_vp.glsl */
 148:
10000: #line 10000 // vertexSimple_vp.glsl
10001: /*
10002: ===========================================================================
10003: Copyright (C) 2010 Robert Beckebans<EMAIL_ADDRESS>10004:
10005: This file is part of XreaL source code.
10006:
10007: XreaL source code is free software; you can redistribute it
10008: and/or modify it under the terms of the GNU General Public License as
10009: published by the Free Software Foundation; either version 2 of the License,
10010: or (at your option) any later version.
10011:
10012: XreaL source code is distributed in the hope that it will be
10013: useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
10014: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
10015: GNU General Public License for more details.
10016:
10017: You should have received a copy of the GNU General Public License
10018: along with XreaL source code; if not, write to the Free Software
10019: Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
10020: ===========================================================================
10021: */
10022: // vertexSimple_vp.glsl - simple vertex fetch
10023:
10024: struct localBasis {
10025: 	vec3 normal;
10026: 	vec3 tangent, binormal;
10027: };
10028:
10029: vec3 QuatTransVec(in vec4 quat, in vec3 vec) {
10030: 	vec3 tmp = 2.0 * cross( quat.xyz, vec );
10031: 	return vec + quat.w * tmp + cross( quat.xyz, tmp );
10032: }
10033:
10034: void QTangentToLocalBasis( in vec4 qtangent, out localBasis LB ) {
10035: 	LB.normal = QuatTransVec( qtangent, vec3( 0.0, 0.0, 1.0 ) );
10036: 	LB.tangent = QuatTransVec( qtangent, vec3( 1.0, 0.0, 0.0 ) );
10037: 	LB.tangent *= sign( qtangent.w );
10038: 	LB.binormal = QuatTransVec( qtangent, vec3( 0.0, 1.0, 0.0 ) );
10039: }
10040:
10041: #if !defined(USE_VERTEX_ANIMATION) && !defined(USE_VERTEX_SKINNING) && !defined(USE_VERTEX_SPRITE)
10042:
10043: IN vec3 attr_Position;
10044: IN vec4 attr_Color;
10045: IN vec4 attr_QTangent;
10046: IN vec4 attr_TexCoord0;
10047:
10048: void VertexFetch(out vec4 position,
10049: 		 out localBasis normalBasis,
10050: 		 out vec4 color,
10051: 		 out vec2 texCoord,
10052: 		 out vec2 lmCoord)
10053: {
10054: 	position = vec4( attr_Position, 1.0 );
10055: 	QTangentToLocalBasis( attr_QTangent, normalBasis );
10056: 	color    = attr_Color;
10057: 	texCoord = attr_TexCoord0.xy;
10058: 	lmCoord  = attr_TexCoord0.zw;
10059: }
10060: #endif
  25: #line 25
20000: #line 20000 // vertexSkinning_vp.glsl
20001: /*
20002: ===========================================================================
20003: Copyright (C) 2009-2011 Robert Beckebans<EMAIL_ADDRESS>20004:
20005: This file is part of XreaL source code.
20006:
20007: XreaL source code is free software; you can redistribute it
20008: and/or modify it under the terms of the GNU General Public License as
20009: published by the Free Software Foundation; either version 2 of the License,
20010: or (at your option) any later version.
20011:
20012: XreaL source code is distributed in the hope that it will be
20013: useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
20014: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
20015: GNU General Public License for more details.
20016:
20017: You should have received a copy of the GNU General Public License
20018: along with XreaL source code; if not, write to the Free Software
20019: Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
20020: ===========================================================================
20021: */
20022: // vertexSkinning_vp.glsl - GPU vertex skinning for skeletal meshes
20023:
20024: #if defined(USE_VERTEX_SKINNING)
20025:
20026: IN vec3 attr_Position;
20027: IN vec2 attr_TexCoord0;
20028: IN vec4 attr_Color;
20029: IN vec4 attr_QTangent;
20030: IN vec4 attr_BoneFactors;
20031:
20032: // even elements are rotation quat, odd elements are translation + scale (in .w)
20033:
20034: void VertexFetch(out vec4 position,
20035: 		 out localBasis LB,
20036: 		 out vec4 color,
20037: 		 out vec2 texCoord,
20038: 		 out vec2 lmCoord)
20039: {
20040: 	const float scale = 1.0 / 256.0;
20041: 	const float weightScale = 1.0 / 255.0;
20042: 	localBasis inLB;
20043:
20044: 	// Unpack data from "bone factors". This used to have the index in the high byte and the weight
20045: 	// in the low byte, which may seem a bit more logical, but it triggered issues with some
20046: 	// Nvidia shader compilers (https://github.com/DaemonEngine/Daemon/issues/472).
20047: 	vec4 ipart = floor( attr_BoneFactors * scale );
20048: 	vec4 fpart = attr_BoneFactors * scale - ipart;
20049: 	// idx = 2 times the original bone index (the index input to boneFactor)
20050: 	ivec4 idx = ivec4( fpart * 512.0 );
20051: 	vec4 weights = ipart * weightScale;
20052:
20053: 	vec4 quat = u_Bones[ idx.x ];
20054: 	vec4 trans = u_Bones[ idx.x + 1 ];
20055:
20056: 	QTangentToLocalBasis( attr_QTangent, inLB );
20057:
20058: 	position.xyz = weights.x * (QuatTransVec( quat, attr_Position ) * trans.w + trans.xyz);
20059: 	LB.normal = weights.x * (QuatTransVec( quat, inLB.normal ));
20060: 	LB.tangent = weights.x * (QuatTransVec( quat, inLB.tangent ));
20061: 	LB.binormal = weights.x * (QuatTransVec( quat, inLB.binormal ));
20062:
20063: 	quat = u_Bones[ idx.y ];
20064: 	trans = u_Bones[ idx.y + 1 ];
20065:
20066: 	position.xyz += weights.y * (QuatTransVec( quat, attr_Position ) * trans.w + trans.xyz);
20067: 	LB.normal += weights.y * (QuatTransVec( quat, inLB.normal ));
20068: 	LB.tangent += weights.y * (QuatTransVec( quat, inLB.tangent ));
20069: 	LB.binormal += weights.y * (QuatTransVec( quat, inLB.binormal ));
20070:
20071: 	quat = u_Bones[ idx.z ];
20072: 	trans = u_Bones[ idx.z + 1 ];
20073:
20074: 	position.xyz += weights.z * (QuatTransVec( quat, attr_Position ) * trans.w + trans.xyz);
20075: 	LB.normal += weights.z * (QuatTransVec( quat, inLB.normal ));
20076: 	LB.tangent += weights.z * (QuatTransVec( quat, inLB.tangent ));
20077: 	LB.binormal += weights.z * (QuatTransVec( quat, inLB.binormal ));
20078:
20079: 	quat = u_Bones[ idx.w ];
20080: 	trans = u_Bones[ idx.w + 1 ];
20081:
20082: 	position.xyz += weights.w * (QuatTransVec( quat, attr_Position ) * trans.w + trans.xyz);
20083: 	LB.normal += weights.w * (QuatTransVec( quat, inLB.normal ));
20084: 	LB.tangent += weights.w * (QuatTransVec( quat, inLB.tangent ));
20085: 	LB.binormal += weights.w * (QuatTransVec( quat, inLB.binormal ));
20086:
20087: 	position.w = 1.0;
20088: 	LB.normal   = normalize(LB.normal);
20089: 	LB.tangent  = normalize(LB.tangent);
20090: 	LB.binormal = normalize(LB.binormal);
20091:
20092: 	color    = attr_Color;
20093: 	texCoord = attr_TexCoord0;
20094: 	lmCoord  = attr_TexCoord0;
20095: }
20096: #endif
  26: #line 26
30000: #line 30000 // vertexAnimation_vp.glsl
30001: /*
30002: ===========================================================================
30003: Copyright (C) 2010 Robert Beckebans<EMAIL_ADDRESS>30004:
30005: This file is part of XreaL source code.
30006:
30007: XreaL source code is free software; you can redistribute it
30008: and/or modify it under the terms of the GNU General Public License as
30009: published by the Free Software Foundation; either version 2 of the License,
30010: or (at your option) any later version.
30011:
30012: XreaL source code is distributed in the hope that it will be
30013: useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
30014: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
30015: GNU General Public License for more details.
30016:
30017: You should have received a copy of the GNU General Public License
30018: along with XreaL source code; if not, write to the Free Software
30019: Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
30020: ===========================================================================
30021: */
30022: // vertexAnimation_vp.glsl - interpolates .md3/.mdc vertex animations
30023:
30024: #if defined(USE_VERTEX_ANIMATION)
30025:
30026: IN vec3 attr_Position;
30027: IN vec4 attr_Color;
30028: IN vec4 attr_QTangent;
30029: IN vec2 attr_TexCoord0;
30030: IN vec3 attr_Position2;
30031: IN vec4 attr_QTangent2;
30032:
30033:
30034: void VertexAnimation_P_N(	vec3 fromPosition, vec3 toPosition,
30035: 				vec4 fromQTangent, vec4 toQTangent,
30036: 				float frac,
30037: 				inout vec4 position, inout vec3 normal)
30038: {
30039: 	vec3 fromNormal = QuatTransVec( fromQTangent, vec3( 0.0, 0.0, 1.0 ) );
30040: 	vec3 toNormal = QuatTransVec( toQTangent, vec3( 0.0, 0.0, 1.0 ) );
30041:
30042: 	position.xyz = 512.0 * mix(fromPosition, toPosition, frac);
30043: 	position.w = 1;
30044:
30045: 	normal = normalize(mix(fromNormal, toNormal, frac));
30046: }
30047:
30048: void VertexFetch(out vec4 position,
30049: 		 out localBasis LB,
30050: 		 out vec4 color,
30051: 		 out vec2 texCoord,
30052: 		 out vec2 lmCoord)
30053: {
30054: 	localBasis fromLB, toLB;
30055:
30056: 	QTangentToLocalBasis( attr_QTangent, fromLB );
30057: 	QTangentToLocalBasis( attr_QTangent2, toLB );
30058:
30059: 	position.xyz = 512.0 * mix(attr_Position, attr_Position2, u_VertexInterpolation);
30060: 	position.w = 1;
30061:
30062: 	LB.normal = normalize(mix(fromLB.normal, toLB.normal, u_VertexInterpolation));
30063: 	LB.tangent = normalize(mix(fromLB.tangent, toLB.tangent, u_VertexInterpolation));
30064: 	LB.binormal = normalize(mix(fromLB.binormal, toLB.binormal, u_VertexInterpolation));
30065:
30066: 	color    = attr_Color;
30067: 	texCoord = attr_TexCoord0;
30068: 	lmCoord  = attr_TexCoord0;
30069: }
30070: #endif
  27: #line 27
40000: #line 40000 // vertexSprite_vp.glsl
40001: /*
40002: ===========================================================================
40003: Copyright (C) 2010 Robert Beckebans<EMAIL_ADDRESS>40004:
40005: This file is part of XreaL source code.
40006:
40007: XreaL source code is free software; you can redistribute it
40008: and/or modify it under the terms of the GNU General Public License as
40009: published by the Free Software Foundation; either version 2 of the License,
40010: or (at your option) any later version.
40011:
40012: XreaL source code is distributed in the hope that it will be
40013: useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
40014: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
40015: GNU General Public License for more details.
40016:
40017: You should have received a copy of the GNU General Public License
40018: along with XreaL source code; if not, write to the Free Software
40019: Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
40020: ===========================================================================
40021: */
40022: // vertexSprite_vp.glsl - sprite vertex fetch
40023:
40024: #if defined(USE_VERTEX_SPRITE)
40025:
40026: IN vec3 attr_Position;
40027: IN vec4 attr_Color;
40028: IN vec4 attr_TexCoord0;
40029: IN vec4 attr_Orientation;
40030:
40031:
40032: float           depthScale;
40033:
40034: void VertexFetch(out vec4 position,
40035: 		 out localBasis normalBasis,
40036: 		 out vec4 color,
40037: 		 out vec2 texCoord,
40038: 		 out vec2 lmCoord)
40039: {
40040: 	vec2 corner;
40041: 	float radius = attr_Orientation.w;
40042: 	vec3 normal = normalize( u_ViewOrigin - attr_Position ), up, left;
40043: 	float s, c; // sin & cos of rotation factor
40044:
40045: 	corner = sign( attr_TexCoord0.zw );
40046:
40047: 	if( radius <= 0.0 ) {
40048: 		// autosprite2 mode, attr_Orientation.xyz contains the up-vector
40049: 		up = attr_Orientation.xyz;
40050: 		left = radius * normalize( cross( up, normal ) );
40051: 		position = vec4( attr_Position + corner.y * left, 1.0 );
40052: 	} else {
40053: 		// autosprite mode, attr_Orientation.x contains the rotation angle
40054: 		left = normalize( cross( u_ViewUp, normal ) );
40055: 		up = cross( left, normal );
40056:
40057: 		s = radius * sin( radians( attr_Orientation.x ) );
40058: 		c = radius * cos( radians( attr_Orientation.x ) );
40059:
40060: 		// rotate left and up vectors
40061: 		vec3 leftOrig = left;
40062: 		left = c * left + s * up;
40063: 		up = c * up - s * leftOrig;
40064:
40065: 		left *= corner.x;
40066: 		up *= corner.y;
40067:
40068: 		position = vec4( attr_Position + left + up, 1.0 );
40069: 	}
40070: 	normalBasis.normal = normal;
40071: 	normalBasis.tangent = normalize( up );
40072: 	normalBasis.binormal = normalize( left );
40073:
40074: 	texCoord = 0.5 * corner + 0.5; //attr_TexCoord0.xy;
40075: 	lmCoord  = abs( attr_TexCoord0.zw );
40076: 	color    = attr_Color;
40077:
40078: 	depthScale = 2.0 * radius;
40079: }
40080: #endif
  28: #line 28
  29:
  30: #if !defined(USE_VERTEX_SPRITE)
  31: #endif
  32:
  33:
  34: #if defined(USE_TCGEN_ENVIRONMENT)
  35: #endif
  36:
  37: #if defined(USE_VERTEX_SPRITE)
  38: OUT(smooth) vec2	var_FadeDepth;
  39: #elif defined(USE_DEPTH_FADE)
  40: OUT(smooth) vec2	var_FadeDepth;
  41: #endif
  42:
  43: OUT(smooth) vec2	var_TexCoords;
  44: OUT(smooth) vec4	var_Color;
  45:
  46: void DeformVertex( inout vec4 pos,
  47: 		   inout vec3 normal,
  48: 		   inout vec2 st,
  49: 		   inout vec4 color,
  50: 		   in    float time);
  51:
  52: void	main()
  53: {
50000: #line 50000 // material_vp.glsl
50001: /*
50002: ===========================================================================
50003:
50004: Daemon BSD Source Code
50005: Copyright (c) 2024 Daemon Developers
50006: All rights reserved.
50007:
50008: This file is part of the Daemon BSD Source Code (Daemon Source Code).
50009:
50010: Redistribution and use in source and binary forms, with or without
50011: modification, are permitted provided that the following conditions are met:
50012:     * Redistributions of source code must retain the above copyright
50013:       notice, this list of conditions and the following disclaimer.
50014:     * Redistributions in binary form must reproduce the above copyright
50015:       notice, this list of conditions and the following disclaimer in the
50016:       documentation and/or other materials provided with the distribution.
50017:     * Neither the name of the Daemon developers nor the
50018:       names of its contributors may be used to endorse or promote products
50019:       derived from this software without specific prior written permission.
50020:
50021: THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
50022: ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
50023: WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
50024: DISCLAIMED. IN NO EVENT SHALL DAEMON DEVELOPERS BE LIABLE FOR ANY
50025: DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
50026: (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
50027: LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
50028: ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
50029: (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50030: SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
50031:
50032: ===========================================================================
50033: */
50034:
50035: /* material_vp.glsl */
50036:
50037: #if defined(USE_MATERIAL_SYSTEM)
50038:
50039: #ifdef HAVE_ARB_shader_draw_parameters
50040:   in_drawID = drawID;
50041:   in_baseInstance = baseInstance;
50042: #endif // !HAVE_ARB_shader_draw_parameters
50043:
50044: #endif // !USE_MATERIAL_SYSTEM
  62: #line 62
  63:
  64: 	vec4 position;
  65: 	localBasis LB;
  66: 	vec4 color;
  67: 	vec2 texCoord, lmCoord;
  68:
  69: 	VertexFetch( position, LB, color, texCoord, lmCoord );
  70: 	color = color * u_ColorModulate + u_Color;
  71:
  72: 	DeformVertex( position,
  73: 		      LB.normal,
  74: 		      texCoord,
  75: 		      color,
  76: 		      u_Time);
  77:
  78: 	// transform vertex position into homogenous clip-space
  79: 	gl_Position = u_ModelViewProjectionMatrix * position;
  80:
  81: 	// transform texcoords
  82: #if defined(USE_TCGEN_ENVIRONMENT)
  83: 	{
  84: 		// TODO: Explain why only the rotational part of u_ModelMatrix is relevant
  85: 		position.xyz = mat3(u_ModelMatrix) * position.xyz;
  86:
  87: 		vec3 viewer = normalize(u_ViewOrigin - position.xyz);
  88:
  89: 		float d = dot(LB.normal, viewer);
  90:
  91: 		vec3 reflected = LB.normal * 2.0 * d - viewer;
  92:
  93: 		var_TexCoords = 0.5 + vec2(0.5, -0.5) * reflected.yz;
  94: 	}
  95: #elif defined(USE_TCGEN_LIGHTMAP)
  96: 	var_TexCoords = (u_TextureMatrix * vec4(lmCoord, 0.0, 1.0)).xy;
  97: #else
  98: 	var_TexCoords = (u_TextureMatrix * vec4(texCoord, 0.0, 1.0)).xy;
  99: #endif
 100:
 101: #if defined(USE_DEPTH_FADE)
 102: 	// compute z of end of fading effect
 103: 	vec4 fadeDepth = u_ModelViewProjectionMatrix * (position - u_DepthScale * vec4(LB.normal, 0.0));
 104: 	var_FadeDepth = fadeDepth.zw;
 105: #elif defined(USE_VERTEX_SPRITE)
 106: 	vec4 fadeDepth = u_ModelViewProjectionMatrix * (position - depthScale * vec4(LB.normal, 0.0));
 107: 	var_FadeDepth = fadeDepth.zw;
 108: #endif
 109:
 110: 	var_Color = color;
 111: }

^3Warn: Compile log:
ERROR: 4:19: 'readonly' : reserved word
ERROR: 4:19: 'readonly' : syntax error syntax error

This seems to be a driver bug. Also, it's reporting an entirely incorrect line number.

I tried requesting OpenGL 4.6 (using r_glMajorVersion and r_glMinorVersion) and it compiled! (And worked OK if I turned off GPU occlusion culling.)
So either we're lacking some extension, or Intel's extension loading is buggy. Might we consider requesting a higher core profile than 3.2 when material system is enabled, instead of loading extensions a la carte?

Intel's extension loading is buggy

This is the case, since the extension spec states:

(insert after third paragraph, p. 73) The memory qualifiers "coherent",
"volatile", "restrict", "readonly", and "writeonly" may be used in the
declaration of buffer variables (i.e., members of shader storage blocks).
...
Additionally, memory qualifiers may also be used in the declaration of
shader storage blocks.

Might we consider requesting a higher core profile than 3.2 when material system is enabled, instead of loading extensions a la carte?

I think that's reasonable, yeah. Another thing that might work is changing the shader version.

This should work:
diff --git a/src/engine/sys/sdl_glimp.cpp b/src/engine/sys/sdl_glimp.cpp
index e9a28ab38..ad317b247 100644
--- a/src/engine/sys/sdl_glimp.cpp
+++ b/src/engine/sys/sdl_glimp.cpp
@@ -1021,13 +1021,15 @@ static rserr_t GLimp_ValidateBestContext(
 	For information about core, compatibility and forward profiles,
 	see https://www.khronos.org/opengl/wiki/OpenGL_Context */

+	bool testGL46 = r_materialSystem.Get();
+
 	struct {
 		int major;
 		int minor;
 		glProfile profile;
 		bool testByDefault;
 	} glSupportArray[] {
-		{ 4, 6, glProfile::CORE, false },
+		{ 4, 6, glProfile::CORE, testGL46 },
 		{ 4, 5, glProfile::CORE, false },
 		{ 4, 4, glProfile::CORE, false },
 		{ 4, 3, glProfile::CORE, false },

I recommended to use a dedicated bool variable and not read the cvar directly because the line setting this bool may be extended later with an option to disable the workaround.

Lower profiles than 4.6 (4.3 and up can be worth checking) might work too.

Yes, I would be curious to know the lowest version that works, and if an extension is missing, that may help to identify it.

4.2 is the lowest that works.

Is there any reason to not just always try to get the highest context version, regardless of driver/hardware? Would also help prevent shader compile errors that happen when core functionality has different naming then the one in an extension, at an earlier point.

I remember @slipher reporting that testing all versions until the one that works can be very slow on Windows.

We can save it in a cvar so it will just go into autogen.cfg.


refactor(german language): fixed some mistakes in the German language

Fixed "requestPermissions", "deafenedTitle", "soundEffectSettings", "followsYou",
"blockedFromRoomUsers", "micPermissionBanner", "createRoomModal", "editRoomModal", "profileModal",
"roomSettingsModal", "inviteButton", "followingOnline", "upcomingRoomsCard", "debugAudio",
"tommorow", "today", "roomChat" in the german language

The scopes are still wrong. try chore(kibbeh): fixed some mistakes in the German language


[Question] Are there any Java examples available on how to use this?

Unfortunately, I have never tried this in Java.


dart run build_runner build complains about missing dart.ui

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
Install the example from edgedb-dart
https://github.com/edgedb/edgedb-dart/tree/main
Get the apps from pubspec.
Run the command in the command line
dart run build_runner build
This should create the necessary files....
Expected results
Create necessary files
Actual results
This is the final error message:
[SEVERE] Failed to precompile build script .dart_tool/build/entrypoint/build.dart.
This is likely caused by a misconfigured builder definition.
This is the information at the beginning:
[WARNING] ../../flutter/packages/flutter/lib/src/services/asset_bundle.dart:8:8: Error: Dart library 'dart:ui' is not available on this platform.
import 'dart:ui' as ui;
^
Context: The unavailable library 'dart:ui' is imported through these packages:
build.dart => package:edgedb => package:flutter => dart:ui
...

Detailed import paths for (some of) the these imports:
build.dart => package:edgedb/edgeql_codegen.dart => package:edgedb/src/base_proto.dart => package:edgedb/src/connect_config.dart => package:edgedb/src/credentials.dart => package:flutter/services.dart => package:flutter/src/services/asset_bundle.dart => dart:ui

Code sample
Code sample
This is the build.dart  contents
final _builders = <_i1.BuilderApplication>[
  _i1.apply(
    r'edgedb:edgeql_codegen',
    [_i2.edgeqlCodegenBuilder],
    _i1.toDependentsOf(r'edgedb'),
    hideOutput: false,
  )
];
void main(
  List<String> args, [
  _i3.SendPort? sendPort,
]) async {
  var result = await _i4.run(
    args,
    _builders,
  );
  sendPort?.send(result);
  _i5.exitCode = result;
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
dart run build_runner build

[INFO] Entrypoint:Generating build script...
[INFO] Entrypoint:Generating build script completed, took 226ms

[INFO] Bootstrap:Precompiling build script......
[WARNING] Bootstrap:
../../flutter/packages/flutter/lib/src/foundation/debug.dart:5:8: Error: Dart library 'dart:ui' is not available on this platform.
import 'dart:ui' as ui show Brightness;
       ^
../../flutter/packages/flutter/lib/src/foundation/debug.dart:11:1: Error: Dart library 'dart:ui' is not available on this platform.
export 'dart:ui' show Brightness;
^
../../flutter/packages/flutter/lib/src/foundation/debug.dart:115:1: Error: Type 'ui.Brightness' not found.
ui.Brightness? debugBrightnessOverride;
^^^^^^^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/debug.dart:115:4: Error: 'Brightness' isn't a type.
ui.Brightness? debugBrightnessOverride;
   ^^^^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:5:8: Error: Dart library 'dart:ui' is not available on this platform.
import 'dart:ui' as ui;
       ^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:311:23: Error: Type 'ui.Image' not found.
  void _imageOnCreate(ui.Image image) {
                      ^^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:319:25: Error: Type 'ui.Picture' not found.
  void _pictureOnCreate(ui.Picture picture) {
                        ^^^^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:327:24: Error: Type 'ui.Image' not found.
  void _imageOnDispose(ui.Image image) {
                       ^^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:333:26: Error: Type 'ui.Picture' not found.
  void _pictureOnDispose(ui.Picture picture) {
                         ^^^^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:290:15: Error: Undefined name 'Image'.
    assert(ui.Image.onCreate == null);
              ^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:291:15: Error: Undefined name 'Image'.
    assert(ui.Image.onDispose == null);
              ^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:292:15: Error: Undefined name 'Picture'.
    assert(ui.Picture.onCreate == null);
              ^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:293:15: Error: Undefined name 'Picture'.
    assert(ui.Picture.onDispose == null);
              ^^^^^^^
../../flutter/packages/flutter/lib/src/foundation/memory_allocations.dart:294:8: Error: Undefined name 'Image'.
    ui.Image.onCreate = _imageOnCreate;
       ^^^^^
                                    ^^^^^^^^^^^^

<SNIP>

Cut due to size...

<SNAP>
       ^
../../flutter/packages/flutter/lib/src/services/font_loader.dart:77:12: Error: The method 'loadFontFromList' isn't defined for the class 'FontLoader'.
 - 'FontLoader' is from 'package:flutter/src/services/font_loader.dart' ('../../flutter/packages/flutter/lib/src/services/font_loader.dart').
Try correcting the name to the name of an existing method, or defining a method named 'loadFontFromList'.
    return loadFontFromList(list, fontFamily: family);
           ^^^^^^^^^^^^^^^^
[INFO] Bootstrap:Precompiling build script... completed, took 1.3s

[SEVERE] Bootstrap:
Failed to precompile build script .dart_tool/build/entrypoint/build.dart.
This is likely caused by a misconfigured builder definition.

</details>

### Flutter Doctor output

<details><summary>Doctor output</summary>

```console
[✓] Flutter (Channel stable, 3.10.6, on macOS 13.4.1 22F770820d darwin-arm64, locale de-DE)
    • Flutter version 3.10.6 on channel stable at /Users/torstenbecker/Documents/Prog/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f468f3366c (4 weeks ago), 2023-07-12 15:19:05 -0700
    • Engine revision cdbeda788a
    • Dart version 3.0.6
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/torstenbecker/Library/Android/sdk
    • Platform android-33, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] Connected device (3 available)
    • iPhone 14 Pro Max (mobile) • BA2A0524-9D19-493E-8682-EDF4B98DB84D • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)            • macos                                • darwin-arm64   • macOS 13.4.1 22F770820d darwin-arm64
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome 115.0.5790.170

[✓] Network resources
    • All expected network resources are available.

• No issues found!

I can't replicate your error. Here are the steps I did:

Replication attempt
user@hostname T % git clone https://github.com/edgedb/edgedb-dart/
Cloning into 'edgedb-dart'...
remote: Enumerating objects: 648, done.
remote: Counting objects: 100% (282/282), done.
remote: Compressing objects: 100% (197/197), done.
remote: Total 648 (delta 147), reused 165 (delta 84), pack-reused 366
Receiving objects: 100% (648/648), 273.63 KiB | 562.00 KiB/s, done.
Resolving deltas: 100% (343/343), done.
user@hostname T % cd edgedb-dart/example
user@hostname example % echo """final _builders = <_i1.BuilderApplication>[
  _i1.apply(
    r'edgedb:edgeql_codegen',
    [_i2.edgeqlCodegenBuilder],
    _i1.toDependentsOf(r'edgedb'),
    hideOutput: false,
  )
];
void main(
  List<String> args, [
  _i3.SendPort? sendPort,
]) async {
  var result = await _i4.run(
    args,
    _builders,
  );
  sendPort?.send(result);
  _i5.exitCode = result;
}""" > build.dart
user@hostname example %  dart run build_runner build
Resolving dependencies in /private/var/folders/xt/bjlm_q315pq6hlt23zx2z9hh0000gn/T/edgedb-dart/example... (1.8s)
Got dependencies in /private/var/folders/xt/bjlm_q315pq6hlt23zx2z9hh0000gn/T/edgedb-dart/example.
Building package executable... (6.8s)
Built build_runner:build_runner.
[INFO] Generating build script completed, took 422ms
[INFO] Precompiling build script... completed, took 7.4s
[INFO] Building new asset graph completed, took 813ms
[INFO] Checking for unexpected pre-existing outputs. completed, took 1ms
[INFO] Running build completed, took 565ms
[INFO] Caching finalized dependency graph completed, took 31ms
[INFO] Succeeded after 609ms with 1 outputs (1 actions)
user@hostname example %

flutter doctor -v
user@hostname example % flutter doctor -v
[✓] Flutter (Channel stable, 3.10.6, on macOS 13.4.1 22F770820d darwin-x64, locale en-IT)
    • Flutter version 3.10.6 on channel stable at /Users/user/development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f468f3366c (4 weeks ago), 2023-07-12 15:19:05 -0700
    • Engine revision cdbeda788a
    • Dart version 3.0.6
    • DevTools version 2.23.1

Thanks for testing, I need to update the case to get the error. I have gotten the example to work, but a different case is not working.

@tbecker Looks like your code is using dart:ui's classes in Dart project instead of Flutter, see https://github.com/dart-lang/sdk/issues/50854#issuecomment-1366596944.
In case you believe this is a Flutter bug, please provide the different case you mentioned above along with detailed steps to reproduce. Thanks!


get rid of zeroes

if anyone has examples of use cases for zeroes literal, please post them here and I'll make sure there is a suggested alternative way to solve that use case.

Not sure its a correct, good or valid use case, but I had this...(diff)
-    var column: [(if (s1len > s2len) s1len else s2len) + 1]usize = zeroes;
+    const maxlen: usize = (if (s1len > s2len) s1len else s2len) + usize(1);
+    var column: [maxlen]usize = []usize{usize(0)} ** maxlen;

Which now gives this error message.
zig build --static --export exe --name test_stringalgo_dbg test_stringalgo.zig
./levenshtein.zig:24:18: error: unable to evaluate constant expression
    var column: [maxlen]usize = []usize{usize(0)} ** maxlen;
                 ^
./levenshtein.zig:24:54: error: unable to evaluate constant expression
    var column: [maxlen]usize = []usize{usize(0)} ** maxlen;
                                                     ^

ah ok there was another change here.
it used to be that the variable declaration syntax would do alloca (increase the stack size by an amount known only at runtime) if the array length was not compile-time known. I decided that this a way to accidentally shoot yourself in the foot, so now you have to explicitly use const column = @alloca(usize, maxlen). Docs for @alloca:

@alloca(inline T: type, count: usize) -> []T
Allocates memory in the stack frame of the caller. This temporary space is
automatically freed when the function that called alloca returns to its caller,
just like other stack variables.
When using this function to allocate memory, you should know the upper bound
of count. Consider putting a constant array on the stack with the upper bound
instead of using alloca. If you do use alloca it is to save a few bytes off
the memory size given that you didn't actually hit your upper bound.
The allocated memory contents are undefined.

The ** operator operates on compile-time known values only, so if maxlen is not compile-time known then you can't use the ** operator. You can do this though:
const mem = @import("std").mem;
mem.set(usize, column, 0);

Is the compiler correct that maxlen is a runtime-known value?

Aye, maxlen is indeed a runtime-known value. I had no particular reason for using zeroes other than convenience I guess (and trying out stuff in langref).
So a const column = @alloca(...) is writeable!? (seems to work with both const and var)
The @alloca thing works fine, so does using a global like below (and possibly allocation of some other sort) obviously one could be a bit more careful.
var globalarray : [512]usize = undefined;
:
: 8< snip 8< and in the fn
    const column = if (maxlen <= 16) @alloca(usize, maxlen) else ([]usize)(globalarray);
    for (column) |*b| {
            *b = 0;
    }

So a const column = @alloca(...) is writeable!? (seems to work with both const and var)

Yeah, seems kind of weird that you can write through const doesn't it?
What's going on is that the concept of "const" is referring to memory range. When you have a const variable, the memory range that contains the variable cannot be changed through that variable.
When you have a pointer, the pointer address itself exists in memory too, and that can be const or not, and then the memory range the pointer points to can be const or not. That's why we have & vs &const and [] vs []const.
So in this example @alloca is returning a []usize which is a mutable slice. However the slice itself is a pointer and a size (16 bytes on x86_64), and those 16 bytes are constant. Meaning you cannot re-assign the column variable to a different slice.
It works with var but in the future I will add an error for using var when you could have used const instead. #224
So it makes sense, but it's still confusing that you can write through a const. So I think this is an argument in favor of using let instead of const (See #181)
Be careful with that globalarray code because that causes problems with thread safety and re-entrancy safety. var array: [512]usize = undefined is allowed inside a function, because 512 is a compile-time known value.