Enabling LTO produces broken extension modules

Nuitka version, full Python version and Platform (Windows, OSX, Linux ...)

$ python -m nuitka --version
<IP_ADDRESS>
Python: 3.6.9 (default, Oct  8 2020, 12:12:24)
Executable: /tmp/nuitka-bug/.venv/bin/python
OS: Linux
Arch: x86_64

How did you install Nuitka and Python (pip, anaconda, deb, rpm, from source,
what is a virtualenv ...), this is very important usually.

Nuitka was installed with pip install nuitka.

If possible please supply a Short, Self Contained, Correct, Example
that demonstrates the issue i.e a small piece of code which reproduces
the issue and can be run with out any other (or as few as possible)
external dependencies.

This can be reproduced by creating a single empty Python file, building it into an extension module with LTO enabled and trying to import the extension module:
$ touch test.py

$ python -m nuitka --module test.py --output-dir=build --lto
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating C source code for backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:WARNING: You are not using ccache.
Nuitka:INFO: Successfully created 'build/test.cpython-36m-x86_64-linux-gnu.so'.

$ cd build && python -c "import test"
Missing decoding for 0

Aborted

Importing the extension module produces an unhelpful error message and aborts the interpreter. Expected behavior would be no output in this case.
If the extension module is compiled without the --lto flag, it can be imported without issues.
Also note that I seem to be able to compile other (larger) projects with LTO just fine. It looks like the issue only shows for some projects, but I couldn't find any reason for this.

If this is a regression (used to work in an earlier version of Nuitka),
please note what you know about that.

This issue did not exist in Nuitka versions prior to 0.6.9. As far as I can tell, <IP_ADDRESS> was the last version that made the above minimal example not crash.

This might be related to #868 as we see the same error message there. Though Conda is not involved in this case.

This might be related to #868 as we see the same error message there. Though Conda is not involved in this case.

That's a big oops for LTO on Linux right there.
I don't think it's related to the mentioned issue, just the same symptom. The constants blob is being added through an object file, with ld directly, and it seems to treat my blob unfairly. I reproduced this on my Debian just now the same. Binaries and non-lto work both well as you report.
The constants blob is not new by itself, but the new code introduced in 0.6.9 might be more sensitive to noticing corruptions. I will have to read up on this. Forcing to use C code for the constants blob data makes it work. Obviously going via the C compiler for data is not good for (ccache) caching, nor any good for compile times, but probably also not too bad.

That's a big oops for LTO on Linux right there.
I don't think it's related to the mentioned issue, just the same symptom. The constants blob is being added through an object file, with ld directly, and it seems to treat my blob unfairly. I reproduced this on my Debian just now the same. Binaries and non-lto work both well as you report.
The constants blob is not new by itself, but the new code introduced in 0.6.9 might be more sensitive to noticing corruptions. I will have to read up on this. Forcing to use C code for the constants blob data makes it work. Obviously going via the C compiler for data is not good for (ccache) caching, nor any good for compile times, but probably also not too bad.

Apply this mini-diff as a workaround:
-    and getLinkerArch(target_arch=target_arch, mingw_mode=mingw_mode) is not None
+    and getLinkerArch(target_arch=target_arch, mingw_mode=mingw_mode) is not None and not lto_mode

Apply this mini-diff as a workaround:
-    and getLinkerArch(target_arch=target_arch, mingw_mode=mingw_mode) is not None
+    and getLinkerArch(target_arch=target_arch, mingw_mode=mingw_mode) is not None and not lto_mode

I also tried clang, but it behaves the same, although I think it's using a different linker.

I also tried clang, but it behaves the same, although I think it's using a different linker.

// We need to make sure this is local to the package, or else it will
// be taken from any external linkage.
#if defined(_NUITKA_MODULE) && !defined(_NUITKA_CONSTANTS_FROM_RESOURCE) && !defined(__cplusplus)
const unsigned char constant_bin[0];
#endif

There is also this, which I am myself confused about now. I believe the symbol ought to not be exported, so this wouldn't have a point, removing that also seems to make it work, so the question becomes why it got added. :)

// We need to make sure this is local to the package, or else it will
// be taken from any external linkage.
#if defined(_NUITKA_MODULE) && !defined(_NUITKA_CONSTANTS_FROM_RESOURCE) && !defined(__cplusplus)
const unsigned char constant_bin[0];
#endif

There is also this, which I am myself confused about now. I believe the symbol ought to not be exported, so this wouldn't have a point, removing that also seems to make it work, so the question becomes why it got added. :)

So, the commit that introduced this first was done for Fedora 32 which apparently found something wrong with it, and that is a workaround that can't quite last. I am assuming the LTO then has a duplicate symbol, where this one wins, and it should not.

So, the commit that introduced this first was done for Fedora 32 which apparently found something wrong with it, and that is a workaround that can't quite last. I am assuming the LTO then has a duplicate symbol, where this one wins, and it should not.

Yes, I think I am seeing that constant_bin becomes a symbol to be exported, which can't be right, if that declaration is removed.
I guess, I need to convince the linker to include an object, but without exporting it there.

Yes, I think I am seeing that constant_bin becomes a symbol to be exported, which can't be right, if that declaration is removed.
I guess, I need to convince the linker to include an object, but without exporting it there.

I found this, it seems to do everything but MSVC, which is fine, we can use the resource mode there too:
https://github.com/graphitemaster/incbin
It seems this way, the inline assembly is used and a binary blob include can be done there. I will need to convince myself if the symbol is exported, but I bet, that can be addressed.

I found this, it seems to do everything but MSVC, which is fine, we can use the resource mode there too:
https://github.com/graphitemaster/incbin
It seems this way, the inline assembly is used and a binary blob include can be done there. I will need to convince myself if the symbol is exported, but I bet, that can be addressed.

This is fantastic, it seems to be already working, and it's only less code, esp. it doesn't need to know linker arch, and should be equally fast. It ought to work on non-Linux as well, probably allowing me to remove the source code mode for constants.

This is fantastic, it seems to be already working, and it's only less code, esp. it doesn't need to know linker arch, and should be equally fast. It ought to work on non-Linux as well, probably allowing me to remove the source code mode for constants.

And on the upside, the created binary is smaller, the symbol becomes global, but this might be something patchable. I am going to submit this to my private and public CI, lets see what comes up. Fedora 33 didn't show the problem of duplicate symbols, but the OBS build with Fedora 32 might, but I doubt it can happen this way.

And on the upside, the created binary is smaller, the symbol becomes global, but this might be something patchable. I am going to submit this to my private and public CI, lets see what comes up. Fedora 33 didn't show the problem of duplicate symbols, but the OBS build with Fedora 32 might, but I doubt it can happen this way.

The Windows gcc (MinGW64) works with it nicely, clangcl.exe does not, seemingly too MSVC compatible there, I didn't try the normal clang.exe yet on Windows yet, but I doubt it can be an issue. I noticed though, that for some reason on Windows the resource mode binary is smaller, which is strange, so I am not using it there yet.

The Windows gcc (MinGW64) works with it nicely, clangcl.exe does not, seemingly too MSVC compatible there, I didn't try the normal clang.exe yet on Windows yet, but I doubt it can be an issue. I noticed though, that for some reason on Windows the resource mode binary is smaller, which is strange, so I am not using it there yet.

In the CI, on RHEL 6 and older, the constants blob wasn't working anymore for some test cases, which is strange. I have therefore made a gcc version check for >= 7 to enable it now, but I am also adding a verfication at run time, taking a CRC32 of the constants blob, to see if it has been tampered with. That should catch #868 and alikes sooner, as well as this one.
Not sure if what I have seen there is a incbin bug, it definitely is one if the constants binary mismatches for some cases, but for now, I am aiming at getting it to work for most cases, continuing to use the old code with old gcc, but then it would surely be nice to get to use incbin with everything.

In the CI, on RHEL 6 and older, the constants blob wasn't working anymore for some test cases, which is strange. I have therefore made a gcc version check for >= 7 to enable it now, but I am also adding a verfication at run time, taking a CRC32 of the constants blob, to see if it has been tampered with. That should catch #868 and alikes sooner, as well as this one.
Not sure if what I have seen there is a incbin bug, it definitely is one if the constants binary mismatches for some cases, but for now, I am aiming at getting it to work for most cases, continuing to use the old code with old gcc, but then it would surely be nice to get to use incbin with everything.

I had to update it again, since incbin is leading to exported symbols, for 2 extension modules, the second one would load the wrong one. Got around that by adding a local feature for incbin, now it's working.
Also where it wasn't working, it seems that an existing bug was caught and that got fixed too. For case collisions of module names Nuitka was using seemingly the wrong constants blob on case sensitive file systems. Not sure really how this worked befoe, or why it wasn't occuring before, but it had nothing to do with incbin at all.
So incbin is now used for everything but Windows, where resources are still used.
There is an impact of the C code that does the incbin not being cacheable and a need to pass the valiue via define. I guess, the code to get the data included ought to be generated instead, and be a separate module, then this won't be an issue and there is no need to pass anything.

I had to update it again, since incbin is leading to exported symbols, for 2 extension modules, the second one would load the wrong one. Got around that by adding a local feature for incbin, now it's working.
Also where it wasn't working, it seems that an existing bug was caught and that got fixed too. For case collisions of module names Nuitka was using seemingly the wrong constants blob on case sensitive file systems. Not sure really how this worked befoe, or why it wasn't occuring before, but it had nothing to do with incbin at all.
So incbin is now used for everything but Windows, where resources are still used.
There is an impact of the C code that does the incbin not being cacheable and a need to pass the valiue via define. I guess, the code to get the data included ought to be generated instead, and be a separate module, then this won't be an issue and there is no need to pass anything.

I changed it again, generating a small C file that will be uncachable. Newer ccache detects that, but older on e.g. RHEL 6 does not, so this is actively disabled for just that one file, which is a good idea anyway, but it cost me really a lot of time to debug why it was using corrupt constant blobs, that had good CRC32 checks.
Anyway, the improvement is real and will be in the next release that is bound to happen any minute now, running final pre-release tests to see what has issues before making actual release.

I changed it again, generating a small C file that will be uncachable. Newer ccache detects that, but older on e.g. RHEL 6 does not, so this is actively disabled for just that one file, which is a good idea anyway, but it cost me really a lot of time to debug why it was using corrupt constant blobs, that had good CRC32 checks.
Anyway, the improvement is real and will be in the next release that is bound to happen any minute now, running final pre-release tests to see what has issues before making actual release.

Released as part of 0.6.11 just now.

Released as part of 0.6.11 just now.


Add support for IOS_SIMULATOR_HOME environment variable in IOSSimulat…

…or.logFilePath
flutter_tools can be run on environments where the user's HOME directory
is not the root of the iOS simulators' configs. This change adds support
for such environments by allowing the caller to set the simulator root
directory via an environment variable.


Impl. np.asarray(literal)

As the title suggests. This pull request implements np.asarray(Literal).

Thanks very much for the PR, I've queued it for review.

It seems that np.asarray(大处着眼，小处着手。🐍⚡) is being interpreted as having dtype <U13 on Windows rather than <U12.

This is being caused by the snake emoji, which is the only 4-byte character in that string.  NumPy has a UCS-4 item size for unicode (as it has to in order to have fixed-size encoding), but on Windows it computes the dtype length assuming UTF-16 (?) encoding.  Looks like an obscure bug in NumPy (it isn't wrong, just slightly memory inefficient), so it would be fair to relax the test strictness.

This PR is ready for a round of reviews

cc @hameerabbasi

I don't think I'm qualified to review this one. ☹

Thanks for the review @stuartarchibald. I've replaced the current implementation by a simpler one as you suggested.

Thanks for the review @stuartarchibald


link to the latest docs will be broker when deprecated = false in config.toml

This is a Bug Report
Problem:
Link to the latest docs will be broken when deprecated = false in config.toml [https://github.com/kubernetes/website/blob/release-1.14/config.toml#L73]
This  issue is resolved in the #13649
Proposed Solution:
{{ T "latest_version" }}

Use .Site.Params
Page to Update:
https://github.com/kubernetes/website/blob/release-1.14/layouts/shortcodes/deprecationwarning.html#L8

I am working on this issue

@rahulkrishnanfs Can you close the issue if this is fixed?

/close


remote 'foxx-manager upgrade app_path_or_zip_file' fails

Hello,
Running windows 2.5.0 arangodb server (Windows 7 Professional 64 bits).
Client computer is Windows 8.1 PRO 64 bits, with ArangoDb 2.5.1 64 bits (although testing with same version of server in client gives the same error)
Remote arangodb has authetication enabled, both on _system and actual database used to deploy foxx application.
When trying to upgrade an already installed foxx application on remote server it fails with error " must be a valid directory name".

actual parameters of the cmd are:
cmd> foxx-manager --server.endpoint tcp://<IP_ADDRESS>:8529 --server.database mydb --server.password mypwd upgrade %TEMP%\updated.zip /myappmountpoint

uninstalling foxx app from the command line is ok:
cmd> foxx-manager --server.endpoint tcp://<IP_ADDRESS>:8529 --server.database mydb --server.password mypwd uninstall /myappmountpoint

re-installing it with same zip file is also ok:
cmd> foxx-manager --server.endpoint tcp://<IP_ADDRESS>:8529 --server.database mydb --server.password mypwd install %TEMP%\updated.zip /myappmountpoint

We have checked temp directories in both server and client machine, assuring everybody has permissions to downloads, uploads, app and zip folders, but not sure if that was the issue.
any hints?
thanks

Can you try to have a look at the logfile of arangod so we can find out whether this is in arangosh/foxmanager or the server itself?
Most probably you should also see this error in the windows eventlog (if)

Hello dothebart,
arangod logfile show nothing (at least i cant find anything that points to a problem on the server).
also windows eventlog show nothing about this.
logfile is being generated with default options.
I have tried to inspect tcp data sent/received comparing install vs upgrade, and it seems that foxmanager does not send the zipped file to the server (install does), but can not asure this 100% since i'm too new to arango ;).
(with arangosh we get same result as with foxxmanager)

ah, 2.5.0 on the server, 2.5.1 on the client - misread that.
2.5.1 has lots of windows polishing - the eventlog too.
if you copy the zip file to the current directory where you call arangosh - does it fix the issue?

i'm now upgrading server to 2.5.1
will report in some minutes, just see

sadly after upgrading server to 2.5.1 problem remains...
now:

client 2.5.1
server 2.5.1
foxxmanager install local_zip_o_folder /mountpoint   is ok
foxxmanager upgrade local_zip_o_folder /mountpoint   fails

have re-checked arangod.log (now with 2.5.1 server) and windows eventlog, nothing there

Ok, problem confirmed & reproduced; its a windows only problem that somehow didn't occur on our windows CI; fix will be part of the next release 2.5.2

thankyou !!!

any news about this, did it got resolved with 2.5.2?
did see nothing in 2.5.2 changelog about this (can not upgrade to 2.5.2 at the moment, at least not before a few days).
thanks in advance

The fix is part of 2.5.3

Have just upgraded my system to 2.5.3 (Windows x64 binaries, both client PC and server PC) and the problem persists.
I guess that the problem fixed is not really the one reported.
Can this be re-opened?
To get in touch, the problem is that we can not upgrade (via foxx-manager cmd line) a foxx application on our development ArangoDb server from neither of our personal development computers.
We try to do that with the following command line:
cmd> foxx-manager --server.endpoint tcp://<IP_ADDRESS>:8529 --server.database mydb --server.password mypwd upgrade updated.zip /myappmountpoint
Thanks in advance, Arango is awesome!!!

Ok, the error message is not helpfull at all.
Running the server in console mode with debug log gave the real failure; adding the path to the zip file fixed the incident.
running it with .\updated.zip instead of updated.zip fixes the incident for me.

Hello,
I have tried using .\updated.zip as stated, but had the same error.
Even used full path (although local to foxx-manager binary computer), and i'm not sure what do you mean when talking about path has to be relative to the running arangod server, updated.zip and .\updated.zip are both relative, on the other hand, how can i tell from client computer what the full path will be on remote computer (arangod server)?
I'm a bit confused, install command gives no error at all with exactly the same parameters.
Sorry if i can not able to describe all the details you would need.
What can i do to help you in finding the real problem? I would do whatever you need.
Thanks

Hi,
i think we first have to clarify some things ;)
First: the install command of foxx manager is designed to work with absolute or relative paths on the machine you execute it. You do not need to know anything about the server, which actually could be running on a different OS). However we often use the servers-console internally which lead to the confusion with @dothebart comment ;)
Second: It can only recognize paths starting with one of the following in windows (C could be any drive as well):
C:\
.\
\
..\

This means the correct command for you should be
cmd> foxx-manager --server.endpoint tcp://<IP_ADDRESS>:8529 --server.database mydb --server.password mypwd upgrade .\updated.zip /myappmountpoint
However there is a bug in the upgrade procedure which actually did not handle local files...
Sorry for the inconvenience i will fix it in a second.

Should now be fixed in 2.5 and level and will be included in the next releases.

On 05/12/15 02:29, Michael Hackstein wrote:

Should now be fixed in 2.5 and level and will be included in the next
releases.
—
Reply to this email directly or view it on GitHub
https://github.com/arangodb/arangodb/issues/1292#issuecomment-101081155.

Computa sagt naain:
willi@bruecklinux:~/src/devel$ git checkout
js/client/modules/org/arangodb/foxx/manager.js
willi@bruecklinux:~/src/devel$ git pull --all
willi@bruecklinux:~/src/devel$ bin/foxx-manager  --server.database
_system upgrade v1.2.1.zip /blarg
 must be a valid directory name
POST /_db/_system/_admin/foxx/upgrade HTTP/1.1.
Host: <IP_ADDRESS>.
Connection: Keep-Alive.
User-Agent: ArangoDB.
Accept-Encoding: deflate.
Authorization: Basic cm9vdDo=.
Content-Length: 72.
.
{"appInfo":"v1.2.1.zip","mount":"/blarg","options":{"configuration":{}}}

T <IP_ADDRESS>:8529 -> <IP_ADDRESS>:35388 [AP]
HTTP/1.1 400 Bad Request.
Server: ArangoDB.
Connection: Keep-Alive.
Content-Type: application/json; charset=utf-8.
Content-Length: 94.
.
{"error":true,"code":400,"errorNum":10,"errorMessage":" must be a
valid directory name"}

Still not resolved; reproducible on linux. The zip is not uploaded to the server, but a path passed on from foxx-manager to arangod.

#eeb7d82e625b1ecd344bfa24a733fa4b2a99859f finally resolves this issue for me; @agomezalvic  can you try to replace your js/client/modules/org/arangodb/foxx/manager.js by https://raw.githubusercontent.com/arangodb/arangodb/2.5/js/client/modules/org/arangodb/foxx/manager.js ?

Hi,
I'll try it a bit later (now have some meetings...).
Should i replace client computer only, only server one or both?
thanks.

its for the foxx manager client; so on which ever computer you run it, replace that file.

Perfect @dothebart
I finllay could test it from client computer and upgrade completed successfully.
thanks!!!

fixed with 2.5.5


Input Source mode HDMI settings

Can we get some additional "input source" settings added for generic HDMI inputs.
For example:

HDMI1
HDMI2
HDMI3
HDMI4
HDMI5

Moving this issue to the public bug tracker: https://issuetracker.google.com/issues/136103095


react 15.2.1 Warning: Unknown prop expandable on component

Problem description
warning.js:44 Warning: Unknown prop expandable on  tag. Remove this prop from the element. For details, see https://fb.me/react-unknown-prop
in div (created by CardText)
in CardText (created by PageDetails)
in div (created by Card)
in div (created by Paper)
in Paper (created by Card)
in Card (created by PageDetails)
in PageDetails (created by KObjectPage)
in div (created by Paper)
in Paper (created by KObjectPage)
in KObjectPage (created by Connect(KObjectPage))
in Connect(KObjectPage) (created by RouterContext)
in div (created by Main)
in div (created by Main)
in MuiThemeProvider (created by Main)
in Main (created by Connect(Main))
in Connect(Main) (created by RouterContext)
in RouterContext (created by Router)
in Router
in Provider
Steps to reproduce
import React from 'react';
import {Card, CardActions, CardHeader, CardText} from 'material-ui/Card';
import FlatButton from 'material-ui/FlatButton';

const CardExampleWithoutAvatar = () => (
  <Card>
    <CardHeader
      title="Without Avatar"
      subtitle="Subtitle"
      actAsExpander={true}
      showExpandableButton={true}
    />
    <CardText expandable={true}>
      Lorem ipsum dolor sit amet, consectetur adipiscing elit.
      Donec mattis pretium massa. Aliquam erat volutpat. Nulla facilisi.
      Donec vulputate interdum sollicitudin. Nunc lacinia auctor quam sed pellentesque.
      Aliquam dui mauris, mattis quis lacus id, pellentesque lobortis odio.
    </CardText>
    <CardActions expandable={true}>
      <FlatButton label="Action1" />
      <FlatButton label="Action2" />
    </CardActions>
  </Card>
);
export default CardExampleWithoutAvatar;

Versions

Material-UI: ^0.15.2
React: ^15.2.1
Browser: Chrome Version 51.0.2704.103 (64-bit)

I can't reproduce this issue on the master branch. That should be fixed now.

I received the same warning message with the same version of react and material-ui.

<CardHeader
title={ {moment(taskDay.Start).format('MM/DD/YY HH:mm')} Job No {taskDay.JobAccountCode} }
subtitle={taskDay.JobName}
actAsExpander={true}
showExpandableButton={true}
/>

Warning: Unknown prop expandable on  tag. Remove this prop from the element. For details, see https://fb.me/react-unknown-prop
in div (created by CardText)
in CardText (created by blah blah)


Design a Register Page UI #2 & Login page #3

added login and register.html file

added login and register.html file

Hello @sadvika05 , This is a mern stack project. Have you worked on any mern stack project previously?

@Trisha-tech yes i have worked on MERN stack projects before
I have made a MERN-stack E-commerce website for my college project
and few other like youtube clone

@Trisha-tech
As you did not mention to make these register and login UI using react or in ejs format
i have just made them in basic HTML

@Trisha-tech As you did not mention to make these register and login UI using react or in ejs format i have just made them in basic HTML

Okay, no problem
Just do one thing, add these files like you make components in react js.
If you face any difficulty, ping me.

okay I'll make react components individually
and I have a doubt does this project come under GSSoC'24 because I don't see any labels

okay I'll make react components individually and I have a doubt does this project come under GSSoC'24 because I don't see any labels

Yes, this project comes under GSSOC'24. Now labels will reflect.

@Trisha-tech
is this issue got closed?
because I can see login and signup pages are already present right now
did they get added or
now i made it should i make a pull request my committing??


Use ubuntu to build????

Can I use ubuntu to build images??
When ever i try i always end up with this error :
ric@ric-ubuntu:~/crochet-freebsd$ sudo ./crochet.sh -b RaspberryPi
Starting at Mon Oct 13 09:07:43 IST 2014
Board: RaspberryPi
Source version is: unknown
./crochet.sh: 1: ./crochet.sh: /usr/bin/grep: not found
Building FreeBSD version:
Image name is:
/home/ric/crochet-freebsd/work/FreeBSD-armv6--RPI-B.img
Can't find appropriate FreeBSD xdev tools.
Tested: armv6-freebsd-cc
If you have FreeBSD-CURRENT sources in /usr/src, you can build these with the following command:
cd /usr/src && sudo make XDEV=arm XDEV_ARCH=armv6 WITH_GCC=1 WITH_GCC_BOOTSTRAP=1 WITHOUT_CLANG=1 WITHOUT_CLANG_BOOTSTRAP=1 WITHOUT_CLANG_IS_CC=1 xdev
Run this script again after you have the xdev tools installed.
ric@ric-ubuntu:~/crochet-freebsd$

This is also my question.
I have downloaded the freebsd source code, now I want to create a disk image from it. How can I do it?

Instructions:  https://github.com/freebsd/crochet/wiki/How-to-Build-a-Disk-Image

try on FreeBSD 9-STABLE.

@blacknes Please do not recommend such an old version on FreeBSD. 10.2 or HEAD would be best.


DELIA-44142: wpeframework crash from WifiManager plugin getSignalData()

Reason for change: wpeframework crash from WifiManager plugin getSignalData()
Test Procedure: Build and test based on README.md
Risks: Low
Signed-off-by: Mariusz Strozynski<EMAIL_ADDRESS>

Changes to topic/xione_mvp has to be cherry picked from master
https://github.com/rdkcentral/rdkservices/pull/85 was cherry picked to topic/xione_mvp


Printing from Fastmate issues

When logged into Fastmail in Safari, a print of an email will get:
A header left-justified, containing the subject, and right-justified, with the time it was printed
A footer left-justified, with the actual Fastmail URL of the message, and right-justified, with the page count/total pages.
When Fastmate 1.64 prints, there are no headers and footers.
The margins are different from the Safari print as well.
Why are there no headers and footers?

The first time you select Print from an email in Fastmate, nothing happens.  The second time, it displays the dialog.

If the resulting print would be less than a single page, Fastmate adds an additional blank page.

Hi! Thanks for reporting. Sadly, since WKWebView (the modern web view which Fastmail uses, and also Safari) does not support printing for third party apps, Fastmate falls back to UIWebView when printing. So practically when Fastmail detects a print command, it creates a totally different web view internally, loads the same page, and then sends a print command to it. That's the why of it - the print will never exactly match Safari until Apple allows WKWebView to print in third-party apps. Also I never print myself so I don't notice when Fastmate doesn't keep up, so please keep reporting!
I think most of those can be fixed though. I do set some margins manually and can look into matching those of Safari. As for headers and footers, I may be able to emulate those as well but not sure. I'll need to look into the paging issue as well but since I have no printer I can't verify - do you see the extra blank page in the print preview window too?

@posguy99 Hi again! I tried to update the print styling to look close to Safari, adding similar headers and footers. Also should fix the bug where it doesn't show up on the first time.
However, I cannot reproduce the last one (that fastmate adds an extra page if it's less than a single page).
Can you please try this binary and see if it fixes your problems? I cannot properly test since I don't own a printer. Thanks!
Fastmate-1.6.6-beta1.zip

Ok, 1.6.6-beta1 has headers and footers now, and it doesn't fail to print the first time you try to print.  But it still prints extra pages (that now have headers and footers ^_^ )
I note that if I print an email of only a few lines, I do not get the extra page, while if the email extends into, maybe, the bottom third of the page, I get the second blank page.  So it's email-dependent.  Is there some setting wrt the frame the UIWebView is being rendered into that makes the frame extend into the second page, even though there's no content?

I’m not sure actually, but that’s good information. I will try with some medium size emails. Can you verify that if you print to PDF you get the extra blank page there or not? I can only test with PDF printing and would like to understand if it’s different there

Yes, you still get the extra page.

Thanks!
Also, since I need to render the new texts manually we could change them. Is email URL and title useful to have or is there maybe something better to put there?

Fixed in v1.6.6


Update Title Case in Start Contributing Section of README

👋 Welcome!
Welcome to Chimoney's open-source community! This issue is specifically designed for first-time contributors, so if you're new to open source or haven't contributed to our codebase before, you're in the right place. We're here to help you take your first steps and guide you through the process of making your first contribution.
What You'll Need to Know
You don't need any prior knowledge or experience to work on this issue. We're here to support you every step of the way, so don't hesitate to ask questions if you're unsure about anything.
Problem
In the ReadMe file, the titles e.g. As a Developer are all in uppercase. We need them updated to sentence case
📋 Step by Step

Claim this issue: Comment below to claim this issue. If someone else has already claimed it, feel free to ask if they've started working on it or if they need any help.

Update the code: In lines 38, 45 and 50 convert the titles to sentence case.

Commit your changes: Once you've made the necessary updates, commit your changes and then submit a pull request so we can review your changes and merge them into the main codebase.

💬 Keep Us Updated
We encourage you to keep us updated on your progress. If you encounter any challenges or need assistance, don't hesitate to ask for help. We're here to support you throughout the process.
👥 Collaboration
If you see someone else working on this issue or if you'd like to collaborate with others, feel free to reach out and offer your assistance. Collaboration is key to our success as a community!
❓ Questions?
If you have any questions or need clarification on anything, please leave a comment below. We're here to help!

can i  solve this issues
]

can u check pull request

Hi @pojith
Unfortunately, this issue is reserved. Kindly feel free to pick any other issue without the reserve tag.

Hi @phyleria @Khadeeejah
can I work on this?


Processing stops on date field

type id: 2 (date), 4 bytes
An option to skip date fields  would be nice.
(I only saw date fields filled with zero bytes, so cannot check decoding)
Observations on the file format

Apparently before each field value there is a byte with 00 or 01 as value.

(The first of these in a row is mentioned in
https://github.com/linville/pydbisam/blob/main/NOTES.md
under Row Definition:   (Offset: 0x19 Size: 2  Description: Trailing \x01 marker),
however it is only 1 byte)

When the byte is 00, the field value is filled with zeros.
My guess: 00 encodes not set (NULL) values to distinguish from actual zeros.

Under File format: the "MD5 Signature" is identical among a set of files I saw. So probably not a checksum.

Thanks for the report!
I added support for date fields. It's unverified as I don't have any databases with that type to verify against but I think it's likely to work. I'll release a new version here shortly, let me know here if you experience any issues.
I merged in the empty-field-marker branch (just forgot it was out there waiting).
Regarding the 16-byte field in the header, yes I'm seeing that of the few databases I have, they all have the same value too. I've changed the notes to be more ambiguous since I don't have any good docs on what it is. A windows utility that displays the databases refers to that field as "File Signature" so I've just updated the file notes to reflect as-such.

Released v1.1.0.


secure_element i2c mutex protection (IDFGH-5505)

Hello,
I would want to use the library secure_element atecc608_ecdsa at the same time of some I2C sensors.
Most of them could be find on this repo: esp-idf-lib.
¿Could you add mutex usage by the library?
A quick fix is to modify hal_esp32_i2c.c functions but, as this library consists to build secure elements, could be better if it can be updated without making this changes every time.
I wrote a post some weeks ago but without any answer from Espressif: link
Thanks.

@iscapla
It is not very clear from your report as where do you expect mutex to be added. Can you please elaborate, please point to specific code pointers for same?

@iscapla Thanks for reporting, would you please help share if any more details for the feature request? Thanks.

Hello @mahavirj @Alvin1Zhang,
In my custom board development I have ESP32 all possible pins connected to other peripherals so It is impossible to use 2 independent I2C buses: one for sensors, other for atecc608 secure element.
As the code is implemented using different threads is will be problems to access to the same I2C bus for "sensor thread" and "netif thread". For that reason I need a mutex to implement an atomic use of the bus.
As sensors library implements a common mutex for all sensors, my request refers to use that same mutex to block I2C usage for 'secure element'.
I ignore where could be best location to use the mutex inside the "secure element" code without causing any troubles.
To sum up my request.....
As atecc608 I2C connection could be shared with other devices from different threads, it is needed to provide a previously created mutex (same for all devices on the same bus). These mutex has to be reserved before I2C transmission and release after all.
Thanks so much.

@iscapla

As sensors library implements a common mutex for all sensors, my request refers to use that same mutex to block I2C usage for 'secure element'.

I am not sure how we can help here, as this is third party library. Probably you can modify application to have higher layer synchronization between using "secure element" access APIs and "sensor library" APIs. This way you should be able to handle this issue entirely in application layer code itself.
If you still think that I2C driver provided in ESP-IDF ("secure_element" I2C HAL is based on top of this) is missing out on some synchronization then please point specific case, we will evaluate further.

A possible solution could be:
(on file: hal_esp32_i2c.c)
ATCA_STATUS hal_i2c_send(ATCAIface iface, uint8_t address, uint8_t *txdata, int txlength)
{
    ATCAIfaceCfg *cfg = iface->mIfaceCFG;
    esp_err_t rc;

    if (!cfg)
    {
        return ATCA_BAD_PARAM;
    }

    //---------- RESERVE MUTEX
    if (!xSemaphoreTake(cfg->atcai2c.lock, pdMS_TO_TICKS(CONFIG_I2CDEV_LOCK_TIMEOUT))){
        return ESP_ERR_TIMEOUT; //or ATCA_COMM_FAIL
    }
    //------------------------

    //ESP_LOGD(TAG, "txdata: %p , txlength: %d", txdata, txlength);
    //ESP_LOG_BUFFER_HEXDUMP(TAG, txdata, txlength, 3);

    i2c_cmd_handle_t cmd = i2c_cmd_link_create();
    (void)i2c_master_start(cmd);
    (void)i2c_master_write_byte(cmd, address | I2C_MASTER_WRITE, ACK_CHECK_EN);
    (void)i2c_master_write(cmd, txdata, txlength, ACK_CHECK_EN);
    (void)i2c_master_stop(cmd);
    rc = i2c_master_cmd_begin(cfg->atcai2c.bus, cmd, 10);
    (void)i2c_cmd_link_delete(cmd);

   //---------- RELEASE MUTEX
   xSemaphoreGive(cfg->atcai2c.lock);
   //------------------------

    if (ESP_OK != rc)
    {
        return ATCA_COMM_FAIL;
    }
    else
    {
        return ATCA_SUCCESS;
    }
}

Here we expect that cfg->atcai2c has another component of type mutex initialized by User application.
I can't handle this scenario on application layer because in most of the cases, user hasn't got the option to know when data is sent or received over the socket. Example: when MQTT (esp32 implementation) send the PINGREQ message.

Hi @iscapla, The I2C driver in the esp-idf handles the i2c bus access with the cmd_mux which is locked when i2c_master_cmd_begin API is called.  so I think adding a separate mutex for I2C bus in secure element shouldn't be required.
Also in your suggested solution you have added the semaphore in the hal_esp32_i2c.c. Does it solve your problem ? I am not quite sure how it would be useful to synchronise the I2C bus access of a third party sensor which doesn't use this semaphore.
Can you provide any sample code with which I would be able to reproduce the issue on my side ?, I think that would help in fixing it.

Hi @AdityaHPatwardhan,
Thanks so much for the clarification.
Just to confirm it, does that mean I2C will always thread safe?
Due to component problems I still didn't receive the prototype so I can't handle this by code. After knowing the scenario, i tried to figure out which will be the behavior of I2C bus when it has more that 1 device connected in a thread environment (I didn't know that cmd_mux had this protection).
On my suggested solution, the third party sensor has to use the same semaphore to work properly. As you mention, it has no sense if they use different ones.

@iscapla Sorry for reverting to this very late. A discussion had been going on internally about the cause of this issue.
As per this https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/i2c.html#_CPPv420i2c_master_cmd_begin10i2c_port_t16i2c_cmd_handle_t10TickType_t The I2C API are not thread safe.
I think the I2C requires the application to handle it.
But since adding a seperate lock in esp-cryptoauthlib would be a problem if it is not used by other peripherals as well!.
Have you already found any solution to this ? or some code to reproduce this problem.

Hello @AdityaHPatwardhan, thanks so much for your update about I2C thread safe limitations.
I don't understand why could be a problem if esp-cryptoauthlib use a mutex witch is not used externally by other libraries.
I didn't try to solve this scenario by code because our current projects doesn't use secure_element (others will) but I did some search over the libraries. Did you explore the idea to extend esp32 with mutex usage?
I don't think, as I wrote in some messages before, hard coding mutex implementation directly on I2C hal function is the best or correct option.
Please, correct me if I am wrong about what am going to write next.
This library use pkcs11 as one of the Public-Key Cryptography methods, where it is declared a control structure called 'pkcs11_lib_ctx'. On this struct, there is a mutex usage and looks connected to freeRTOS function over "ATCA_STATUS hal_lock_mutex(void * pMutex)" on cryptoauthlib/lib/hal/hal_freertos.c file.
Furthermore, function C_Initialize could initialize all the pkcs11 stuff using a CK_C_INITIALIZE_ARGS parameter type where a custom mutex usage could be added.
Is there any configuration macro or initialization point from a esp32 app code to use all this hierarchy?

Hi @iscapla What I meant was,
To synchronise multiple third party variables, we would need all of them to use the same semaphore for blocking the I2C access. Since currently it is not handled in the I2C driver. We may have to handle it in the application layer.
I don't think there is anything similar in case of esp32 that is present at the moment.
I am having discussion with internal team and see if we find any alternate way to resolve your issue.

Hi @AdityaHPatwardhan,
Thank you for the clarification. Please, let me know if you achieve any solution to this scenario. I will continue this thread you if I find any proposal.

Hi @AdityaHPatwardhan,
I'm implementing now a product with LCD via SPI, touch panel via I2C, and IO expansion MCP23017 via I2C.
For touch, I'm using espressif new touch bsp driver (https://github.com/espressif/esp-bsp/tree/master/components/esp_lcd_touch), and MCP23017 share the same I2C port because I don't have more spare pins.
So my team and I do not think that is a good idea to modify the espressif component to implement a lock in an external semaphore to MCP23017 and F5X06 works nicely (they are in different tasks), because espressif's driver will be updated and we need to make some modifications again to follow improvements made on it.
There is a nice guy @Rop Gonggrijp that already make a great job implementing the I2C_Manager https://github.com/ropg/i2c_manager, please take a look.
Considering the nature of I2C interface to share devices on the same interface, we think that the espressif i2c driver should be thread-safe.
Thanks

Hello everyone,
In the new driver for I2C documentation says:
Thread Safety
The factory function i2c_new_master_bus() and i2c_new_slave_device() are guaranteed to be thread safe by the driver, which means, user can call them from different RTOS tasks without protection by extra locks. Other public I2C APIs are not thread safe. which means the user should avoid calling them from multiple tasks, if user strongly needs to call them in multiple tasks, please add extra lock.
Does this mean that new I2C driver is thread safe, or that only i2c_new_master_bus() and i2c_new_slave_device() are thread safe?


Feature/wjiang/get short name work around

The intel compiler does not deallocate the allocated string in the condition block when optimized compiling flag is used. That leads to crash because the memory is used up.
Description

Related Issue

Motivation and Context

How Has This Been Tested?

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Trivial change (affects only documentation or cleanup)

Checklist:

[x] I have tested this change with a run of GEOSgcm (if non-trivial)
[x] I have added one of the required labels (0 diff, 0 diff trivial, 0 diff structural, non 0-diff)
[x] I have updated the CHANGELOG.md accordingly following the style of Keep a Changelog

Hopefully, this patch   can solve your issue temporarily before you upgradeto the latest MAPL.

Should this be "exit" ? @tclune
https://github.com/GEOS-ESM/MAPL/blob/eb3eebb18c47299e3aa87879ac49feeb7a835229/generic/MAPL_Generic.F90#L7087

Should we add if (i == j) cycle between the lines? @tclune
https://github.com/GEOS-ESM/MAPL/blob/eb3eebb18c47299e3aa87879ac49feeb7a835229/generic/MAPL_Generic.F90#L7098-L7099

Should this be "exit" ? @tclune
https://github.com/GEOS-ESM/MAPL/blob/eb3eebb18c47299e3aa87879ac49feeb7a835229/generic/MAPL_Generic.F90#L7087

Really the question should go to @atrayano    I'm wary of changing the logic in such a fundamental way.  Why do you think it should be exit?

Should we add if (i == j) cycle between the lines? @tclune
https://github.com/GEOS-ESM/MAPL/blob/eb3eebb18c47299e3aa87879ac49feeb7a835229/generic/MAPL_Generic.F90#L7098-L7099

Again really a question for @atrayano

This passed the GCM. Once the CI is happy, I'll merge and make a dev to main pr for release.


Correct and enhancement

Page says "When you turn this feature on, three new log files are created" but then lists four log files.  Also, can you explain the details of what additional logging is captured when turning on "additional logging" on the data gateway and where it is located?  Is that info included in the Gateway Performance PBI template file?  Thanks!
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4a144b27-04b6-5d42-9c9f-79bb4cc6730b
Version Independent ID: 7b00f680-f3a8-3502-94b5-5ca37d36f432
Content: Monitor and optimize on-premises data gateway performance
Content Source: data-integration/gateway/service-gateway-performance.md
Product: on-premises-data-gateway
Technology: ****
GitHub Login: @arthiriyer
Microsoft Alias: arthii

Thanks @jamserra for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary. #please-close


Missing Image on M01, Uni6, Step4

Module: 01
Lab/Demo: Unit6
Task: 04
Step: 03
Description of issue
In the 1st lab task 4, step 3, it shows the [T] to copy/paste to the lab for a note, so students are copying this and expecting a result instead of just doing what's there. I think we need to replace that to something easier to follow or at least remove the [T]
Both, step 3 under, first under task 4 and the next step (called "connect to the Test VMs using RDP") have the same issue. There is a .png image that was supposed to be present instead.
Repro steps:

[T] Copy/paste is unique to lab hosters. This feedback has been shared with the lab hoster Skillable.


⚠ Bofrost.at has degraded performance

In ccc1f20, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1462 ms

Resolved: Bofrost.at performance has improved in 81486ea after 10 minutes.


Make counter records be in outputs

Closes #57

The CAPTURE, OFFSET, SCALE, UNITS are in PARAMETERS if the field is not a COUNTER:

and in OUTPUTS if the field is a COUNTER`:

The pos_out pvi screens now have a button to open the table, and CAPTURE, OFFSET, SCALE, UNITS have been put in a new CAPTURE group.

The table has been given a name field

This requires the following changes to pvi (and merge of the device-ref branch):

A way to change this to True
https://github.com/epics-containers/pvi/blob/6a51e6728857a8cd36415afbe03bd6e71aeee032/src/pvi/_format/screen.py#L431

A way to ignore this assert so the "Name" column can be added
https://github.com/epics-containers/pvi/blob/6a51e6728857a8cd36415afbe03bd6e71aeee032/src/pvi/_format/screen.py#L416-L418

A way for the columns to be aligned given the non-pv device name cell

TOP.bob now correctly displays buttons (which work for navigation).

Changed the table back to what it was before for this branch

New group and index.bob:


Add RepLLaMA style models

Adds RepLLaMA models and Promptriever (paper on arxiv tomorrow) models as MTEB models. Verified they reproduced. Subtly, the training code adds an extra space as the prefix that I figured out the hard way several months ago.
Checklist

[x] Run tests locally to make sure nothing is broken using make test.
[x] Run the formatter to format the code using make lint.

Adding a model checklist

[x] I have filled out the ModelMeta object to the extent possible
[x] I have ensured that my model can be loaded using

[x] mteb.get_model(model_name, revision_id) and
[] mteb.get_model_meta(model_name, revision_id)

[x] I have tested the implementation works on a representative set of tasks.

mteb.get_model_meta doesn't seem to work, it gives me
AttributeError: 'ModelMeta' object has no attribute 'encode'

But perhaps I'm doing something wrong @KennethEnevoldsen? Do I use it in the same way as the second line below (e.g. model = mteb.get_model_meta("samaya-ai/promptriever-llama2-7b-v1")? Or perhaps it's because it's a peft model?
Also, weirdly it runs the encoding twice when I use this:
import mteb
model = mteb.get_model("samaya-ai/promptriever-llama2-7b-v1")
tasks = mteb.get_tasks(tasks=["SciFact"], languages=["eng"])
evaluation = mteb.MTEB(tasks=tasks)
evaluation.run(model, batch_size=16)

Tested out mteb.get_model_meta("samaya-ai/promptriever-llama2-7b-v1") can't seem to reproduce the error.
Wanted to test out the double encode as well but can't access the model.

Tested out mteb.get_model_meta("samaya-ai/promptriever-llama2-7b-v1") can't seem to reproduce the error.

Ah sorry, it's public now.  And updated the hashes, great idea!

@orionw I can't seem to get reproduce the error with:
mteb.get_model_meta("samaya-ai/promptriever-llama2-7b-v1")
It seems to work just fine on my end.

@KennethEnevoldsen thanks for checking, I must have something weird in my new local environment. Seems good to merge then, assuming the pyproject commit I just added doesn't break things.


Keyboard Issues in ios 8.4

Hi Jesse,
First I'd like to thank you for making such an awesome program. I really appreciate all the work you do.
I'm having one issue though. It is related to #1076, but it isn't exactly the same.
When I close out of the jsqmessages view controller in ios 8.2 and then re-enter the same conversation, everything works perfectly. However, on my iphone, when I close out of the jsqmessages view controller and re-enter, the keyboard no longer has a textview or send button. When I manually drag down the keyboard before leaving the view in ios 8.4, I can re-enter the conversation, and everything will work. It sounds like resignFirstResponder is working in ios 8.2 but not in ios 8.4. Below is a picture of what it looks like when I re-enter. Any ideas?

@psopher I ran into this problem today and took a shot at debugging it. In your View Controller that subclasses JSQMessagesViewController do you override viewDidAppear? If you do make sure you call super inside it! I had the exact same issue as you and this fixed it for me.

thanks @ferrettmerr !


GUI Breaks when moving between screens of different DPI

I have a multimonitor setup with two different DPI screens on Windows 10 Creators Update. When dragging any StaxRip window between screens of different DPI, the GUI breaks and starts position or sizing incorrectly.
Currently using <IP_ADDRESS>

From my current code base dynamic DPI changing is probably very difficult to implement. I could change it that one monitor uses virtual scaling, it's the thing that gives the awful blurry look, only one monitor would use that.

In the next build I changed it using virtual scaling on one monitor, full support is currently not in sight. I hope it's OK, thanks for reporting.

Build is online now.
https://forum.videohelp.com/threads/369913-StaxRip-Encoder-GUI/page6#post2490759


django-unpoly-6 Allow changing the up-target fpr UnpolyForms

Codecov Report

Merging #7 (0eeb989) into main (76cc580) will not change coverage.
The diff coverage is 0.00%.

@@          Coverage Diff          @@
##            main      #7   +/-   ##
=====================================
  Coverage   0.00%   0.00%
=====================================
  Files          5       5
  Lines         56      90   +34
=====================================
- Misses        56      90   +34

Impacted Files
Coverage Δ

django_unpoly/up.py
0.00% <0.00%> (ø)

django_unpoly/middleware.py
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76cc580...0eeb989. Read the comment docs.


OPTIONS issue

I'm using this in conjunction with shelf_rpc, in this manner:
    var handler = const shelf.Pipeline()
        .addMiddleware(shelf.logRequests())
        .addMiddleware(shelf_cors.createCorsHeadersMiddleware())
        .addMiddleware(exceptionHandler())
        .addHandler(root.handler);

When my client makes  an OPTIONS request, I get these header:
Access-Control-Allow-Origin: *
Content-Type: text/plain; charset=utf-8
Date: Tue, 31 May 2016 13:43:45 GMT
Server: dart:io with Shelf
Transfer-Encoding: chunked
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block

This is causing my client requests to fail. If I remove the CORS handler middleware from the pipeline, it sends these on an OPTIONS request:
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json; charset=utf-8
Date: Tue, 31 May 2016 13:44:13 GMT
Expires: 0
Pragma: no-cache
Server: dart:io with Shelf
Transfer-Encoding: chunked
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block

If I perform a normal GET request, they both produce the exact same output.
Note: I'm using the CORS handler to provide CORS for a route other than the RPC, obviously they're being provided by the RPC system already, I was just hoping to be able to apply the CORS middleware globally to cover all my bases. It interfering with the headers in a OPTION response is an issue one way or the other though.

After reading the source code for this, I can see that this behavior is completely intentional, all OPTIONS requests get overwritten by a generic OK message. This basically makes this library useless for anything except the most basic of implementations, OPTIONS requests need to return accurate information from the underlying resource.


Unknown Status from Windows Server 2016

[*] [<IP_ADDRESS>] Sending negotiation request
[*] [<IP_ADDRESS>] Sending client data
[*] [<IP_ADDRESS>] Sending ping packet
[-] [<IP_ADDRESS>] Status Unknown - Response received but length was 0 not 288

The likelihood is this system is not infected. In our test environment, uninfected machines forcibly terminated the connection and infected machines returned a 288 byte response, both with SSL and non-SSL variants.
Others have reported many systems returning 0-byte and 9-byte responses that were not believed to be infected. We have not seen this in our test environment to confirm so we left it officially as "status unknown" with only a 288-byte response being considered infected

Thank you for your clarification.


Web regression: HtmlElementView onPlatformViewCreated fires before HTML element created

Prior to 3.19, when using HtmlElementView's  onPlatformViewCreated callback, you could access the HTML element within the callback:
HtmlElementView(
    viewType: 'myView',
    onPlatformViewCreated: (viewId) async {
        // Should be able to access the created view here
        window.document.getElementById('myViewId');
    }
)

This is failing in Flutter 3.19, as the HTML element has not yet been added to the DOM at the time onPlatformViewCreated is called.
This has broken production code for me, and I believe it will have broken two projects I've previously worked on.
Steps to reproduce
Run this code sample on Flutter web with 3.19.
Expected results: shows a red box (screenshot from 3.16.9):

Actual results: throws Error: TypeError: null: type 'Null' is not a subtype of type 'DivElement' because the div has not yet been added to the DOM.
import 'package:flutter/material.dart';
import 'dart:ui_web' as ui_web;
import 'dart:html' as html;

void main() {
  ui_web.platformViewRegistry.registerViewFactory(
    'myView',
    (int viewId, {Object? params}) {
      return html.DivElement()
        ..id = 'myDivId'
        ..style.width = '100px'
        ..style.height = '100px';
    },
  );

  runApp(
    MaterialApp(
      home: Scaffold(
        body: HtmlElementView(
          viewType: 'myView',
          onPlatformViewCreated: (viewId) async {
            final div = html.window.document.getElementById('myDivId')
                as html.DivElement;

            div.style.backgroundColor = 'red';
          },
        ),
      ),
    ),
  );
}

Logs

Launching lib/main.dart on Chrome in debug mode...
This app is linked to the debug service: ws://<IP_ADDRESS>:53778/20k_NPLe51o=/ws
Debug service listening on ws://<IP_ADDRESS>:53778/20k_NPLe51o=/ws
Connecting to VM Service at ws://<IP_ADDRESS>:53778/20k_NPLe51o=/ws
Error: TypeError: null: type 'Null' is not a subtype of type 'DivElement'
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 297:3  throw_
dart-sdk/lib/_internal/js_shared/lib/rti.dart 1385:3                         _failedAsCheck
dart-sdk/lib/_internal/js_shared/lib/rti.dart 1363:3                         _generalAsCheckImplementation
dart-sdk/lib/_internal/js_shared/lib/rti.dart 1258:30                        _installSpecializedAsCheck
packages/web_platform_view/main.dart 23:17                                   <fn>
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 84:54           runBody
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 127:5           _async
packages/web_platform_view/main.dart 21:35                                   <fn>
packages/flutter/src/widgets/_html_element_view_web.dart 65:30               <fn>
dart-sdk/lib/async/zone.dart 1661:54                                         runUnary
dart-sdk/lib/async/future_impl.dart 162:18                                   handleValue
dart-sdk/lib/async/future_impl.dart 838:44                                   handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13                                   _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5                                    [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7                                    callback
dart-sdk/lib/async/schedule_microtask.dart 40:11                             _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5                              _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7           <fn>

Doctor output
[✓] Flutter (Channel stable, 3.19.1, on macOS 14.3.1 23D60 darwin-arm64, locale en-GB)
    • Flutter version 3.19.1 on channel stable at /Users/tom/.puro/envs/stable/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision abb292a07e (33 hours ago), 2024-02-20 14:35:05 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/tom/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/tom/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.80.0

[✓] Connected device (4 available)
    • Tom’s iTelephone (mobile)  • 00008130-000A4D8820A1401C            • ios            • iOS 17.3.1 21D61
    • iPhone 15 Pro Max (mobile) • 2904497E-B3E2-450A-B364-A5A0B0D0901A • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-17-2 (simulator)
    • macOS (desktop)            • macos                                • darwin-arm64   • macOS 14.3.1 23D60 darwin-arm64
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome 121.0.6167.184

[✓] Network resources
    • All expected network resources are available.

• No issues found!

Reprodudible using the code sample provided above.
I can confirm that this is a regression as the issue does not reproduce on stable 3.16.0

flutter doctor -v
[!] Flutter (Channel [user-branch], 3.16.0, on macOS 14.3.1 23D60 darwin-arm64, locale en-GB)
    ! Flutter version 3.16.0 on channel [user-branch] at /Users/nexus/dev/sdks/flutter_rc/flutter
      Currently on an unknown channel. Run `flutter channel` to switch to an official channel.
      If that doesn't fix the issue, reinstall Flutter by following instructions at https://flutter.dev/docs/get-started/install.
    ! Upstream repository unknown source is not a standard remote.
      Set environment variable "FLUTTER_GIT_URL" to unknown source to dismiss this error.
    • Framework revision db7ef5bf9f (3 months ago), 2023-11-15 11:25:44 -0800
    • Engine revision 74d16627b9
    • Dart version 3.2.0
    • DevTools version 2.28.2
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Users/nexus/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode-15.2.0.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Users/nexus/Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.5)
    • IntelliJ at /Users/nexus/Applications/IntelliJ IDEA Ultimate.app
    • Flutter plugin version 77.2.2
    • Dart plugin version 232.10286

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.82.0

[✓] Connected device (4 available)
    • Nexus (mobile)       • 00008020-001875E83A38002E • ios            • iOS 17.3.1 21D61
    • Dean’s iPad (mobile) • 00008103-000825C811E3401E • ios            • iOS 17.3.1 21D61
    • macOS (desktop)      • macos                     • darwin-arm64   • macOS 14.3.1 23D60 darwin-arm64
    • Chrome (web)         • chrome                    • web-javascript • Google Chrome 121.0.6167.184

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[✓] Flutter (Channel stable, 3.19.1, on macOS 14.3.1 23D60 darwin-arm64, locale en-GB)
    • Flutter version 3.19.1 on channel stable at /Users/nexus/dev/sdks/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision abb292a07e (2 days ago), 2024-02-20 14:35:05 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Users/nexus/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode-15.2.0.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Users/nexus/Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.5)
    • IntelliJ at /Users/nexus/Applications/IntelliJ IDEA Ultimate.app
    • Flutter plugin version 77.2.2
    • Dart plugin version 232.10286

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.82.0

[✓] Connected device (4 available)
    • Nexus (mobile)       • 00008020-001875E83A38002E • ios            • iOS 17.3.1 21D61
    • Dean’s iPad (mobile) • 00008103-000825C811E3401E • ios            • iOS 17.3.1 21D61
    • macOS (desktop)      • macos                     • darwin-arm64   • macOS 14.3.1 23D60 darwin-arm64
    • Chrome (web)         • chrome                    • web-javascript • Google Chrome 121.0.6167.184

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[!] Flutter (Channel master, 3.20.0-11.0.pre.80, on macOS 14.3.1 23D60 darwin-arm64, locale en-GB)
    • Flutter version 3.20.0-11.0.pre.80 on channel master at /Users/nexus/dev/sdks/flutters
    ! Warning: `flutter` on your path resolves to /Users/nexus/dev/sdks/flutter/bin/flutter, which is not inside your current Flutter SDK checkout at /Users/nexus/dev/sdks/flutters. Consider adding /Users/nexus/dev/sdks/flutters/bin to the front of your path.
    ! Warning: `dart` on your path resolves to /Users/nexus/dev/sdks/flutter/bin/dart, which is not inside your current Flutter SDK checkout at /Users/nexus/dev/sdks/flutters. Consider adding /Users/nexus/dev/sdks/flutters/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision efee280cfa (8 hours ago), 2024-02-21 21:44:25 -0500
    • Engine revision 7eeb697687
    • Dart version 3.4.0 (build 3.4.0-163.0.dev)
    • DevTools version 2.33.0-dev.6
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/nexus/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Users/nexus/Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
    • Xcode at /Applications/Xcode-15.2.0.app/Contents/Developer
    • Build 15C500b
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Users/nexus/Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.5)
    • IntelliJ at /Users/nexus/Applications/IntelliJ IDEA Ultimate.app
    • Flutter plugin version 77.2.2
    • Dart plugin version 232.10286

[✓] VS Code (version 1.86.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.82.0

[✓] Connected device (5 available)
    • Nexus (mobile)                  • 00008020-001875E83A38002E • ios            • iOS 17.3.1 21D61
    • Dean’s iPad (mobile)            • 00008103-000825C811E3401E • ios            • iOS 17.3.1 21D61
    • macOS (desktop)                 • macos                     • darwin-arm64   • macOS 14.3.1 23D60 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin         • macOS 14.3.1 23D60 darwin-arm64
    • Chrome (web)                    • chrome                    • web-javascript • Google Chrome 121.0.6167.184

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Same problem, Work with3.16.x, but fail since 3.19.0/3.19.1
But no errors for my own:
widget code that fail :
import 'package:agomsonline/classes/worldwind.class.dart';
import 'package:flutter/widgets.dart';
import 'dart:html' as html;
import 'dart:ui_web';

// ignore: must_be_immutable
class WordWindMap extends StatefulWidget {
  String? divId;

  WordWindMap({
    super.key,
    this.divId,
  }) {
    divId ??= getRandomDivId();
  }

  @override
  State<WordWindMap> createState() => _WordWindMapState();

  String getRandomDivId() {
    return 'worldwind-map-${DateTime.now().millisecondsSinceEpoch}';
  }
}

class _WordWindMapState extends State<WordWindMap> {
  late html.DivElement mapContainer;

  @override
  void initState() {
    super.initState();

    createWorldWindMapContainer(widget.divId!);
  }

  void createWorldWindMapContainer(String containerId) {
    mapContainer = html.DivElement()..id = containerId;

    // Register view without dimensions
    platformViewRegistry.registerViewFactory(
        containerId, (int viewId) => mapContainer);
  }

  void initWorldWind(int value) {
    // Intialize WorldWind
    WorldWindHelper.init(widget.divId!);
  }

  @override
  Widget build(BuildContext context) {
    final size = MediaQuery.of(context).size;

    // Update div dimensions with widget size
    mapContainer.style.width = '${size.width}px';
    mapContainer.style.height = '${size.height}px';

    return HtmlElementView(
      viewType: widget.divId!,
      onPlatformViewCreated: initWorldWind,
    );
  }
}

/cc @ditman

This is a side effect of the multi-view work. The TL;DR is that platform views are no longer immediately injected into the DOM when they're created, but only when they're actually going to be "rendered" somewhere by the engine. Views are "created" at the same time as before, though, so there's a brief moment of time where the view is created but not injected into the DOM yet.
There's a new API to access the view from the onPlatformViewCreated callback:

ui_web.platformViewRegistry.getViewById(int id) (docs) - Returns an Object that can be used alongside JS-interop (cast to Element or any JS Type you expect), for example. We use it in this example.

We also added a new way of creating platform views where registering a whole factory "just to create a div" is no longer necessary:

HtmlElementView.fromTagName (docs), which has an onElementCreated callback, that returns the Object that it creates, so it can be tweaked as needed. We use this for the pointer_interceptor_web.

Apologies for this, nothing failed on our customer testing, but I should have assumed that others were using these APIs like in our old examples.
I hope the workarounds I presented above help!

@ditman
Tried your workaround, but still not work in my case.
My use case is once the div is inserted in DOM, execute a javascript function that populate the div. So something like this :
HtmlElementView.fromTagName(
  tagName: 'div',
  onElementCreated: (element) {
    (element as html.DivElement)
      ..id = 'myOtherDivId'
      ..style.width = '100px'
      ..style.height = '100px'
      ..style.backgroundColor = 'red';

      // Call Javascript function
      context.callMethod('initWorldWind', ['myOtherDivId']);
  },
),

But apparently, the div is still not inserted in the DOM when onElementCreated is fired.

I was experiencing the same issue, but I managed to resolve it by following what @ditman recommended here:

  HtmlElementView.fromTagName(
        tagName: 'div',
        onElementCreated: (Object element) async {

          var div = (element as html.DivElement)
            ..style.width = '100%'
            ..style.height = '100%';

          js.context['PSPDFKit'].callMethod('load', [
            js.JsObject.jsify({
              'document': widget.documentPath,
              'container': div,
            })
          ]);
        });

In this case PSPDFKit accepts both the element id and the element itself.

I'll add some documentation to the HtmlElementView with what we've discussed in this thread, so we can point people to the API docs directly, instead of them having to find this (closed) issue!

Accepting it as a documentation improvement.

Reprioritized because I'm actively expanding the docs of the HtmlElementView right now.

The new documentation is already visible in the main branch API docs:

https://main-api.flutter.dev/flutter/widgets/HtmlElementView-class.html


Remove inline assembly.

This is broken on nightly, since asm has been changed to llvm_asm. But rather
than fixing it, we can simply drop it, because we now have an optimization
barrier that works on stable Rust with no platform dependencies.
Also, update the README and docs to reflect this change.
Closes #70 #67

Can I get a history on the black_box functions and why there were two of them? This all seems valid to me but I don't know why the asm version wasn't removed earlier, looking at this PR I would have expected the asm version to be removed back when the volatile read version was added.

Previously, the non-asm version was a no-op and the protection was only available on nightly with asm. There wasn't any particular reason why the asm version was kept around at that time, so this can be thought of as deferred maintenance.

@hdevalence 👍

Released as 2.2.3.


Add possibility to forward VNC traffic from Safari browsers to selenoid-ui

I have GGR instance which includes several Mac mini’s with selenoid binaries and safari browsers.
On all of them VNC is turned on but I have no possibility to view browser’s image inside selenoid-ui.
It would be great if it was possible.
Thanks!

I have just started this arrangement, too, and encountered the same issues.

@yurii-tikhonenko several news:

You can proxy VNC with Ggr: https://aerokube.com/ggr/latest/#_proxying_vnc_traffic
MacOS is using non-standard VNC implementation, so not sure if this will work with non-Mac clients.

I've tried to proxy vnc with ggr but I had no success with it.
Also I started x11vnc on Mac on different port and proxy it's vnc traffic through ggr but it's also wasn't working. Maybe I'm doing smth wrong, I do not know.
I would appreciate any information that can help :)

@yurii-tikhonenko when proxying VNC URL should be something like: vnc://<remote-host>:<new-port>.


756 automated new business email

This PR establishes automated new business emails with a simple test. It also adds assertions to the developer_mailer_test file
link to issue: https://github.com/joemasilotti/railsdevs.com/issues/765

Hey @joemasilotti, let me know if this looks like a good implementation to you! I left a TODO for the first message test in developers_mailer_test.rb.
I don't love the assert_enqueued_email assertion, but attempting to scope it to a mailer and method was giving me errors. Once we get this nailed down I'm happy to grab the other mailer ticket as well!

Hey @joemasilotti,
Just writing to let you know that I plan to get this wrapped up in the near future. I'm currently on vacation, but brought my laptop and am juggling this PR and interview code challenges

Sounds great! No rush on it and thanks for the update. Let me know if you need any help.

Hey again @accua, two week check in. Do you still have appetite to finish this PR?
If not, no worries. I can take it over the finish line.

Hey Joe! Happy to bring this across the finish line. How does complete by midweek sound?On Feb 26, 2023, at 12:58 PM, Joe Masilotti @.***> wrote:
Hey again @accua, two week check in. Do you still have appetite to finish this PR?
If not, no worries. I can take it over the finish line.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Sounds great to me, thanks! I'm here if you need anything.

Hey @joemasilotti! I had some spare time this evening and went ahead and made the changes listed above. Let me know your thoughts 😄


add slack integration

Coverage remained the same at 83.065% when pulling bd0e4a618dfcd7b6efe46aa1738a8a8b85dcae5a on feature/add-slack-integration into d8200b2ee66f597b46bb17b6a9945a031556a1cd on develop.


changed .now() to .utcnow()

@berendt Sorry this hasn't been high on our radar lately. We are interested in maintaining this still, and I'm going through the pending pull requests now.

@jmunnings Thanks, that is fine for me. I will open PRs for further improvements. Feel free to drop me a mail if you need further support to maintain this repository. At the moment I am working with oandapy and can have a look on it.


fix: update action from deprecated behavior

Fixes the action so GitHub doesn't print a warning about using deprecated behavior

Sorry, I was in no fit state to deal with my GitHub Actions for the past year. I'll try to keep up in the future.
I fixed this myself while finally going over my actions. Thank you very much for your input.

I would've liked if this was merged, but thanks for fixing it i guess
same goes for the pr on the clang action


Update Composer dependencies (2021-05-27-00-10)

Visual regression test passed!
View the visual regression test report


ternary operator examples

https://harness.atlassian.net/browse/CDS-75439
Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the maintainers.
What Type of PR is This?

[x ] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

https://harness.atlassian.net/browse/CDS-75439

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

some large commit created dozens of conflicts. easier for me to create a new branch

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app


Create and test edit_property_field() - a part of create_properties_template()

Signature: edit_property_field(properties: json, field: str, value: str) -> json:
Arguments:

properties: json with properties
field: string with field to edit
value: string with value to add to the field

Returns: json with properties with a field edited
Context: see the create_properties_template diagram
This function edits the field values to the defaults we use.
@signekb: I'm guessing we'll use this several times and might include a edit_property_fields() function - like with create_dir()and create_dirs(). What say you, @lwjohnst86 ?

Is this basically the same thing as https://github.com/seedcase-project/seedcase-sprout/issues/576 ? If so, we could rename edit_resource_properties_field > edit_property_field or edit_properties_field.

@lwjohnst86 ?


Initial SONiC-DASH Test High Level Description (HLD).

This describes the SONiC-DASH Test Reference Architecture, vocabulary and basic guidelines. No actual testing cases or methodology are included.

Merging for forward movement re: Test Framework HLD.  May be updated in the future as we learn more.


Lockfiles debug

Actually an attempt to dig a bit deeper into the #6096 #6135 problems

The env dump in the py3.12 tests went like this, in part ...
2024-09-04T16:44:11.9294931Z nox > python -m pip install --no-deps --editable .
2024-09-04T16:44:18.3399127Z nox > echo '"NOX TEST ENV DUMP == START "'
2024-09-04T16:44:18.3409259Z "NOX TEST ENV DUMP == START "
2024-09-04T16:44:18.3413493Z nox > env
2024-09-04T16:44:18.3423399Z SYSTEMROOT=
2024-09-04T16:44:18.3424153Z SELENIUM_JAR_PATH=/usr/share/java/selenium-server.jar
2024-09-04T16:44:18.3424913Z CONDA=/home/runner/miniconda3
2024-09-04T16:44:18.3425544Z GITHUB_WORKSPACE=/home/runner/work/iris/iris
2024-09-04T16:44:18.3426381Z JAVA_HOME_11_X64=/usr/lib/jvm/temurin-11-jdk-amd64
2024-09-04T16:44:18.3427105Z CONDA_EXE=/home/runner/miniconda3/bin/conda
2024-09-04T16:44:18.3427693Z _CE_M=
2024-09-04T16:44:18.3428105Z ENV_NAME=ci-tests
  . . .
2024-09-04T16:44:18.3490069Z PROJ_DATA=/home/runner/miniconda3/envs/ci-tests/share/proj
 . . .

2024-09-04T16:44:18.3522405Z PROJ_NETWORK=ON
 . . .
2024-09-04T16:44:18.3523626Z NOX_CURRENT_SESSION=tests
2024-09-04T16:44:18.3524201Z TMPDIR=/home/runner/work/iris/iris/.nox/tests/tmp
2024-09-04T16:44:18.3524969Z NPY_DISABLE_CPU_FEATURES=AVX512F,AVX512CD,AVX512_SKX
2024-09-04T16:44:18.3525782Z nox > echo '".. == END NOX TEST ENV DUMP."'
2024-09-04T16:44:18.3526703Z nox > pytest -n auto --cov=lib/iris --cov-report=xml lib/iris/tests
2024-09-04T16:44:18.3527501Z ".. == END NOX TEST ENV DUMP."

So, I have a string suspicion that the PROJ_DATA=/home/runner/miniconda3/envs/ci-tests/share/proj here
is the key problem.
It does seem from the error messages that this key variable is not re-set to the correct path for the installed test environment, as it should have been.  E.G. the message :

We've seen

Also tested : a "minimal" noxfile like this :
import nox

#: Default to reusing any pre-existing nox environments.
nox.options.reuse_existing_virtualenvs = False  # was True for Iris

def show_env(session: nox.sessions.Session):
    session.run("echo", '"NOX TEST LIVE-ENV DUMP == START "', external=True)
    session.run("env", external=True)
    session.run("echo", '".. == END NOX TEST LIVE-ENV DUMP."', external=True)
    session.run("echo", 'NOX TEST session env."', external=True)
    msg = "\n".join(("  " + str(item)) for item in session.env.items())
    print(msg)
    session.run("echo", '.. = END NOX TEST session env."', external=True)

@nox.session(venv_backend="conda")
def basic(session: nox.sessions.Session):
    show_env(session)

@nox.session(venv_backend="conda")
def withproj(session: nox.sessions.Session):
    session.conda_install("proj", "pyproj")
    show_env(session)

Results
Too long+tedious to copy here, but it suffices to say that PROJ_DATA is not set in the environment where proj is installed.
Checking the nox code, I believe that the Session.conda_install() call eventually calls popen, setting any specific variables from session.env, but there is no effort to capture any env changes that the install makes.
I have also checked that the user "conda install" does set env variables, i.e. effectively it actions activation scripts just like "conda activate" does.
So I conclude that nox.Session.conda_install() just does not support additional activation steps.
This is how an unsuitable PROJ_DATA setting manages to upset the installation of a later version of proj.
However, consulting also our SSS modulefiles, which likewise include variables from activation scripts so as to mimic a "conda activate", it seems that PROJ_DATA is quite possibly the only one which is likely to cause trouble in this way.  So in practice it's a marginal problem.
In this specific case, another way to fix it could be to set PROJ_DATA="invalid", since Proj seems to ignore any value which is not a suitable filepath.  I think you can do that by setting session.env["PROJ_DATA"] = "invalid" in the nox code


Remove unused diff_years method

Summary of changes
I think this method is never hit defp diff_years(a, a), do: 0 in the current implementation, so let's remove it.
Checklist

[x] New functions have typespecs, changed functions were updated
[x] Same for documentation, including moduledocs
[x] Tests were added or updated to cover changes
[x] Commits were squashed into a single coherent commit

Coverage increased (+0.07%) to 68.012% when pulling 846d51b124d93f7c8f4f1f55735943e83499878b on treble37:remove-unused-path into 85ee14feed16ac7ce7827ac6bbdb10471c107141 on bitwalker:master.

Why would it never be hit? If the two years are the same value, then the diff should be 0.

Maybe I am misunderstanding something - I run in an IEx shell using the following:
t = ~D[2017-01-01]
t2 = ~D[2017-02-01]
Timex.Comparable.Diff.diff(t, t2, :years)
#=> 0

Because of line 26, defp do_diff(a, a, type),      do: zero(type), wouldn't Date(times) with the same years pattern match on line 26 first and the removed line would never be hit.  Unless there's another path to diff_years that I am not seeing?

Coverage increased (+0.07%) to 68.06% when pulling afc4e2aa52e725a55441958c5151276f63d17124 on treble37:remove-unused-path into c86a96af43cb0052249a4899d988aaa4f665982b on bitwalker:master.

I've merged this manually, thanks!


Form validation

Validation errors of the angular-wysiwyg are not showing as part of the errors of the parent form. Is that the expected behavior? How to get the validation errors from angular-wysiwyg ?

It took a little while, but I've found that if you surround the wysiwyg tag with an ng-form tag you can reference the forms $invalid,
(this assumes that you have added your own validation to ngModelCtrl.$validators)
<div ng-form="exampleForm">
<wysiwyg textarea-name="text" ng-model="data"></wysiwyg>
{{exampleForm.text.$invalid}} this didn't work for me
{{exampleForm.$invalid}} this did work however
</div>


backends/libdile_vt: use separate thread for vsync handling

This lets us increase framerates from ~50 to ~60 FPS (or 40 to 50) when framerate divider is set to 1.
Seems to work fairly well, but needs testing.
NO_VSYNC_THREAD environment variable can be set for testing purposes.
This behaviour is similar to what libvt does internally - they spawn a thread that just waits for vsync and dispatches "texture ready" events, which is then handled in a separate/main thread when calling VT_GenerateTexture, so should be fairly safe.

This seems to be working fairly well for a week.

So do you think completely unlimiting the framerate is a good idea?
Someone on the hyperion-issue Mariotaku created explained that, when using higher framerates hyperiond has more to do so performance will go down and delay will go higher
IIRC he told that using ~15 fps should be enough

Even we don't have to utilize such high frame rate, this would still be useful. It means with same frame rate, TV will take less time to process, thus have lower latency

Okay cool if both of you think that, thank you guys :)

This is not about "unlimiting" the framerate - this is about keeping the framerate pace in sync with system vsync.
If you set framerate to 30fps in configuration it will get limited to 25/30fps. If you set it to 15fps it will get limited to 12.5/15fps with frames with constant delays between each other.
We shouldn't correlate incoming framerate with hyperiond crashes since that was caused by a separate bug. 50/60fps feed works perfectly fine for me, and if anyone wants to limit their framerate we have a function to do so.


Can you tell me how to get the SumMe dataset?

Can you tell me how to get the SumMe dataset?

https://gyglim.github.io/me/vsum/index.html#benchmark

This link is showing "site not found", any other way to get access of this dataset?

fixed again

Sorry for the inconvenience, but it seems that the link above is not working. May I ask if there is another way to access the dataset? Thank you!

Sorry for the inconvenience, but it seems that the link above is not working. May I ask if there is another way to access the dataset? Thank you!

You can find it here:
https://data.vision.ee.ethz.ch/cvl/SumMe/SumMe.zip
The zip contains the following:
demo.m File that shows how to evaluate your summary
GT/ folder containing the human summary selections
videos/ folder with the videos themselves in webm and mp4 (H.264) format
matlab/ code for the evaluation including plotting
python/ basic code for evaluating in python (needs scipy, numpy, matplotlib)
to run the python demo call (from the dataset root):
python python/demo.py


Compiling against a Java library should honor the library API

Consumers of a Java Library should honor its API. In particular:

consumers referencing non exported classes should fail at compile time
unit tests of the library itself should be able to reference internal classes
integration/functional tests should probably not be able to reference internal classes

Some implementation options:

generate stub classes, consumers compile against the stubs
same as 1, but provide a Jar instead of a classes directory
honor the API using a compiler plugin instead

1 and 2 have an impact on performance, because we need to do more work. In particular, compile avoidance already snapshots the classes of a producer by eliminating non relevant parts. But to be able to generate a compile-time error, we need a way to make the compiler only "see" what it is supposed to see. For Jigsaw modules, this is already provided by the compiler itself, but for older JDKs, or non Jigsaw modules, we need a different strategy.

Still relevant


[Dev] Avoid requiring expected error message on third_party tests

Full run of unittest (./build/release/test/unittest "*") is failing on missing error messages for sqllogic tests.
Same on CI for the nightly tests.
Changed suggested by @Mytherin, makes so that external tests are not required to have messages.
Detection is based on file_name starting with "third_party", it's probably not be super robust.
Proper solution would be passing it down from runTest invocation, I can do that if it make more sense.

Thanks! Could we perhaps use the original_sqlite_test property of the SQLLogicTestRunner instead?

Sure, that can double as a check. Later will add it and send it in to duckdb/duckdb.


CLI failing with exit code -14

Describe the bug
When running dependency-check CLI on a linux box (redhat) via a jenkins job, I am getting an exception thrown, and the job is failing with exit code -14. It is scanning a folder that i have filled with jar files (I have used the maven copy dependency plugin to scan the pom and put the files into the folder).
I am also getting the same exit code when trying to run the same CLI via a windows OS on the same codebase (this time though the log does not show an exception being thrown).
Interestingly, even though it is exiting in both cases, it is still generating all the output files - but I don't know if these are complete or not.
** Version of dependency-check used **
CLI version 3.3.4-RELEASE
** Log file **
Logs files in gist:
https://gist.github.com/aoxley/d7da58dfb48b0cde94ccc7e342e1cae4
This includes both what jenkins log, as well as the log generated by CLI on redhat and windows.
To Reproduce
Unfortunately this is a private repo so a little hard to give an exact reproduction step. The command however being run is:
dependency-check.bat --proxyserver <IP_ADDRESS> --proxyport <PORT> --proxyuser <USER> --proxypass <PASSWORD> --project "test project" --scan "C:\Temp\code\target\dependency-check" --out "C:\Temp\code" --format "ALL" --log "C:\Temp\code\target\dependency-check.log"
As mentioned, it is scanning a folder filled with jar files. Hopefully the logs may give some clue as to what is happening.

This is a known issue that could affect all releases prior to 5.0.0. Please upgrade to the latest version of dependency-check. If you must keep using the version of ODC the only way to make it reliable work would be to setup an internal mirror of the NVD: https://github.com/stevespringett/nist-data-mirror
Note - if you are using the Jenkins plugin to import and track the findings you will have to wait a few more weeks while the 5.0.0 version of the Jenkins plugin is released.

Thanks so much jeremy. I'll take a look at upgrading if I can (I was having to play around with a few different components versions to get ones that played nicely).
I'll close this off now.
Cheers.


Use a random port for GraphiQL when the default one is not available

WHY are these changes introduced?
Fixes https://github.com/Shopify/cli/issues/3578
Since GraphiQL was added, you can't run dev multiple times simultaneously without specifying a different GraphiQL port, as we always use the same default value (3457). When you try to run dev a second time, it raises an error:

WHAT is this pull request doing?
If the default port is in use, find an available one instead of raising an error.
How to test your changes?
Run p shopify app dev in two tabs and ensure that GraphiQL works in both places
Measuring impact
How do we know this change was effective? Please choose one:

[x] n/a - this doesn't need measurement, e.g. a linting rule or a bug-fix
[ ] Existing analytics will cater for this addition
[ ] PR includes analytics changes to measure impact

Checklist

[x] I've considered possible cross-platform impacts (Mac, Linux, Windows)
[x] I've considered possible documentation changes
[x] I've made sure that any changes to dev or deploy have been reflected in the internal flowchart.

I'm not sure this is a good idea. Certain features of GraphiQL (query history, session recovery) break if you switch ports, so I'm thinking users should have to do this manually.

I know, but even without history, GrahpiQL keeps working and it's not the main purpose of the dev command. It must be annoying for people having the default port already in use, or when they want to dev two different apps at the same time...
What if we show a warning when the port is random? Something like:
The default port for GraphiQL (3457) is not available, so a random one has been chosen (5754).

If you want to keep your GrahpiQL session, you can manually select a port by passing `--graphiql-port`.

Another alternative would be asking to kill the other process, as we do with the authentication:


Use Microsoft.NET.Compilers.Toolset

This allows us to consume compilers that are newer than what is bundled in the .NET Core SDK. This is valuable during previews because the compiler is building new features we want to take advantage of.
This upgrades our compiler to the version bundled in arcade, which right now is 3.2.0-beta1-19253-08. This should workaround https://github.com/aspnet/AspNetCore-Internal/issues/2476

Have we applied this fix to the other repos? What do we do for AspNetCore which is not hasn't converted yet?

Only to EFCore. This same change wouldn't work  in AspNetCore because it's not on arcade yet

Right, so how should we mitigate the issue in AspNetCore?

The mitigation wasn't as obvious since AspNetCore isn't using the Arcade SDK, but here is the fix: https://github.com/aspnet/AspNetCore/pull/10415


Rewrite polonius-engine

Requires rust-lang/datafrog#50.
Each section ("naive", "initialization", etc.) is now expressed as a Computation, with defined inputs and outputs. Computations are organized into Pipelines, which execute each computation in turn, storing the results in a Db where they are accessible to later computations in the pipeline.
Computations can also dump intermediate results using the (creatively named) Dump struct. A series of Dumpers can be specified for a pipeline, each which will be given access to a type-erased version of the dumped data.
The purpose of this is to support arbitrary configurations of analysis passes, such as skipping move/init analysis by providing an additional "dropped_while_init" input, or the use of alternate solvers (e.g. Soufflé).
For now, I've preserved the existing interface of polonius-engine (Ouput, AllFacts and Algorithm). I expect this to be temporary, but I wanted to take advantage of the existing test machinery and keep my changes localized (relatively speaking).
This adds some additional dependencies (most of which could be removed) and some nightly features (which cannot be, at least not easily). I'm not sure if compiling under the beta toolchain without RUSTC_BOOTSTRAP is a hard requirement, or if it could be relaxed.

I'm not sure what the path is to upstream this. It's too big to review on its own, obviously. I might have to work in my fork for a while.

@ecstatic-morse are you still working on this?


Use platform-specific line separators

I noticed that when I log an assertion error created by Hamcrest that it uses a single line feed character (Unix style end-of-line). Since the rest of my log uses carriage return / line feed pairs (Windows style end-of-line) this will confuse some editors like Vim or Eclipse which generally expect a file to use the same style throughout.
Rather than hardcoding one or the other, the System.lineSeparator() function will return the end of line separator appropriate for the current runtime, i.e. "\r\n" on Windows and "\n" on Unix. This way it will adapt to the Java runtime's platform. And should for some reason a user prefer the other style, they should be able to set the line.separator property at startup.

I've just fixed a problem with TravisCI that stopped the checks running. Can you please rebase this PR?

@tumbarumba Done.