Another potential stack issue

This program works on the iOS i41CX+ emulator.
(The FOCAL program is from http://hp41programs.yolasite.com/roman.php )
E2 is synthetic for 100
E3 is synthetic for 1000, both replaced in the below code.
$ cat ar3.hp41
LBL "A-R"
CLA
LBL 01
1000
X>Y?
GTO 00
"|-M"
GTO 01
LBL 00
CLX
900
X<=Y?
"|-CM"
X>Y?
CLX
500
X<=Y?
"|-D"
X>Y?
CLX
400
X<=Y?
"|-CD"
X>Y?
CLX
LBL 02
100
X>Y?
GTO 00
"|-C"
GTO 02
LBL 00
CLX
90
X<=Y?
"|-XC"
X>Y?
CLX
50
X<=Y?
"|-L"
X>Y?
CLX
40
X<=Y?
"|-XL"
X>Y?
CLX
LBL 03
10
X>Y?
GTO 00
"|-X"
GTO 03
LBL 00
DSE X
X<=Y?
"|-IX"
X>Y?
CLX
5
X<=Y?
"|-V"
X>Y?
CLX
4
X<=Y?
"|-IV"
X>Y?
CLX
LBL 04
1
X>Y?
GTO 00
"|-I"
GTO 04
LBL 00
AVIEW
END
$
$ xrpn -l ar3.hp41
T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 0.0000   L = 0.0000

page 1

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 0.0000   L = 0.0000

l
Line 1: lbl "A-R"

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 0.0000   L = 0.0000

1000

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 1,000.0000   L = 0.0000

s
Line 2: cla

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 1,000.0000   L = 0.0000

s
Line 3: lbl 01

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 1,000.0000   L = 0.0000

s
Line 4: 0

T = 0.0000   α =
Z = 0.0000
Y = 1,000.0000
X = 0.0000   L = 1,000.0000

s
Line 5: xgty?
NO

T = 0.0000   α =
Z = 0.0000
Y = 1,000.0000
X = 0.0000   L = 1,000.0000

s
Line 6: gto 00

T = 0.0000   α =
Z = 0.0000
Y = 1,000.0000
X = 0.0000   L = 1,000.0000

s
Line 11: clx

T = 0.0000   α =
Z = 0.0000
Y = 1,000.0000
X = 0.0000   L = 1,000.0000

s
Line 12:
Traceback (most recent call last):
/home/acser/bin/xrpn:55:in <main>': undefined method downcase' for nil:NilClass (NoMethodError)
$
$

I cannot reproduce the error. What I see is that the program loops indefinitely, though. I can single-step until I get blue in my face... but no error like the one you report (and I simply did a copy-paste of the listing above, loaded with the -l switch, did page 1and started single-stepping with s.

Hmmm. At any rate, this program on the HP 41 stops with the display of the Roman numeral converted from the Arabic numeral.

I did, however discover one thing that must be fixed: Single-stepping conditionals treats those lines as though they are entered manually and only reports "YES" or "NO" and do not advance the program counter appropriately.

Latest update now treats conditionals correctly.

Great, thanks. Let me recheck the code, it is likely a user (programming) error.

I think I got it.
Line 4 works just fine.
However, line 12 puts the number 900 on the stack. However, it shifts the stack up. This is not the HP 41 behavior. When you type a number and do not press the <Enter^> key on the HP 41, it places the number into X. From this point, it's all broken and results in the infinite loop. I think you have to implement typing a number and not pressing <Enter^> but still placing that number onto the stack. Also, you may want to look into the 'clx' command code as that precedes the offending line 12 but not line 4.
$ cat ar4.hp41
LBL "A-R"
" "
LBL 01
1000
X>Y?
GTO 00
"|-M"
GTO 01
LBL 00
CLX
900
X<=Y?
"|-CM"
X>Y?
CLX
500
X<=Y?
"|-D"
X>Y?
CLX
400
X<=Y?
"|-CD"
X>Y?
CLX
LBL 02
100
X>Y?
GTO 00
"|-C"
GTO 02
LBL 00
CLX
90
X<=Y?
"|-XC"
X>Y?
CLX
50
X<=Y?
"|-L"
X>Y?
CLX
40
X<=Y?
"|-XL"
X>Y?
CLX
LBL 03
10
X>Y?
GTO 00
"|-X"
GTO 03
LBL 00
DSE X
X<=Y?
"|-IX"
X>Y?
CLX
5
X<=Y?
"|-V"
X>Y?
CLX
4
X<=Y?
"|-IV"
X>Y?
CLX
LBL 04
1
X>Y?
GTO 00
"|-I"
GTO 04
LBL 00
AVIEW
END
$
$ xrpn -l ar4.hp41
T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 0.0000   L = 0.0000

14

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 14.0000   L = 0.0000

l
Line 1: lbl "XRPN"

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 14.0000   L = 0.0000

page 1

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 14.0000   L = 0.0000

l
Line 1: lbl "A-R"

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 14.0000   L = 0.0000

s
Line 2: " "

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 14.0000   L = 0.0000

s
Line 3: lbl 01

T = 0.0000   α =
Z = 0.0000
Y = 0.0000
X = 14.0000   L = 0.0000

s
Line 4: 1000

T = 0.0000   α =
Z = 0.0000
Y = 14.0000
X = 1,000.0000   L = 14.0000

s
Line 5: xgty?
YES

T = 0.0000   α =
Z = 0.0000
Y = 14.0000
X = 1,000.0000   L = 14.0000

s
Line 6: gto 00

T = 0.0000   α =
Z = 0.0000
Y = 14.0000
X = 1,000.0000   L = 14.0000

s
Line 11: clx

T = 0.0000   α =
Z = 0.0000
Y = 14.0000
X = 0.0000   L = 14.0000

s
Line 12: 900

T = 0.0000   α =
Z = 14.0000
Y = 0.0000
X = 900.0000   L = 0.0000

I was able to reduce the problem to a much simpler example:
$ cat test2.hp41
lbl "TEST2"
1
2
3
4
clx
5
end
$ xrpn -f test2.hp41
T = 2.0000   α =
Z = 3.0000
Y = 0.0000
X = 5.0000   L = 0.0000

The correct stack should not have Y=0. It should be X=5, Y=4, Z=3, T=2.

Check latest (implemented stack lift disabling, something that is triggered with CLX)
We can't have numbers entered without pressing Enter as there is no way to know how many digits the user will want to enter - unless we also implement time travel.

Also implemented the E2, E3, etc mnemonics

E2 and E3 worked but putting 1000 on the stack still intermittently fails.

"putting 1000 on the stack still intermittently fails"
How?

"putting 1000 on the stack still intermittently fails"
How?

"putting 1000 on the stack still intermittently fails"
How?

It shows up as 0. Check the listing.
On Saturday, April 9, 2022, 11:08:27 AM EDT, Geir Isene @.***> wrote:
"putting 1000 on the stack still intermittently fails"
How?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you modified the open/close state.Message ID: @.***>


fix conflict logical in properties config.

Verify whether to repeat in text mode, when adding properties config.

Codecov Report

Merging #2066 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             master    #2066      +/-   ##
============================================
+ Coverage     50.25%   50.27%   +0.02%
- Complexity     1981     1984       +3
============================================
  Files           400      400
  Lines         12349    12353       +4
  Branches       1275     1275
============================================
+ Hits           6206     6211       +5
  Misses         5686     5686
+ Partials        457      456       -1

Impacted Files
Coverage Δ
Complexity Δ

...portal/component/txtresolver/PropertyResolver.java
86.66% <100%> (+0.62%)
44 <2> (+2)
:arrow_up:

...ervice/service/ReleaseMessageServiceWithCache.java
87.2% <0%> (+1.16%)
25% <0%> (+1%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21e76b3...728286a. Read the comment docs.

Coverage increased (+0.01%) to 53.971% when pulling 728286afa14e16265aa9dca9cc7fb160fcece6bc on JaredTan95:property_resolver into 21e76b3d43f455c1e28a9d6626601362cf4997ae on ctripcorp:master.


Command execution randomly hangs when executing within a transaction

.NET Core 3.0 app running on Linux container.
It performs 4 MERGE operations using Upsert extension (uses
ExecuteSqlRawAsync internally) inside a transaction and writes the data into Azure SQL db:
Slightly obfuscated production code:
public async Task PersistAsync(ProcessStatus processStatus)
{
    var strategy = _dbContext.Database.CreateExecutionStrategy();

    await strategy.ExecuteInTransactionAsync(async () => await PersistData(processStatus), () => Task.FromResult(false));
}

private async Task PersistData(ProcessStatus processStatus)
{
    var entities1 = CreateEntity1(processStatus);
    await _dbContext.Entities1.UpsertRange(entities1)
        .On(v => new { v.Id1, v.Id2 })
        .NoUpdate()
        .RunAsync();

    var entities2 = CreateEntity2(processStatus);
    await _dbContext.Entities2.Upsert(entities2)
        .On(v => new { v.Id1, v.Id2 })
        .RunAsync();

    var entities3 = CreateEntity3(processStatus);
    await _dbContext.Entities3.UpsertRange(entities3)
        .On(v => new { v.Id1, v.Id2 })
        .RunAsync();

    var entities4 = CreateEntity4(processStatus);
    await _dbContext.Entities4.UpsertRange(entities4)
        .On(v => new { v.Id1, v.Id2 })
        .RunAsync();

    await _dbContext.SaveChangesAsync();
}

It worked fine while running on Win Server 2016 VM
After moving to AKS we noticed that the app started hanging randomly while persisting one of the entities.
It can successfully process the data 50-150 times before the issue occurs and randomly hangs on one four entities from the code above.
From EF debug logs before it hangs:
[16:34:50 DBG] Creating DbCommand for 'ExecuteNonQuery'.
[16:34:50 DBG] Created DbCommand for 'ExecuteNonQuery' (0ms).
[16:34:50 INF] Executing DbCommand [Parameters=[@p0='?' (DbType = Int32), @p1='?' (Size = 6) (DbType = AnsiString), @p2='?' (Size = 6) (DbType = AnsiString), @p3='?' (DbType = Int32), @p4='?' (Size = 1) (DbType = Byte), @p5='?' (Size = 1) (DbType = Byte), @p6='?' (DbType = Int32), @p7='?' (DbType = Int32), @p8='?' (Size = 8000) (DbType = AnsiString), @p9='?' (Size = 255) (DbType = AnsiString), @p10='?' (DbType = DateTime), @p11='?' (Size = 2) (DbType = AnsiString), @p12='?' (Size = 255) (DbType = AnsiString), @p13='?' (DbType = Int32), @p14='?' (DbType = DateTime)], CommandType='Text', CommandTimeout='30']
MERGE INTO [Entities4] WITH (HOLDLOCK) AS [T] USING ( VALUES (@p0, @p1, @p2, @p3, @p4, @p5, @p6, @p7, @p8, @p9, @p10, @p11, @p12, @p13, @p14) ) AS [S] ([xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [Json], [xxxx], [LastUpdated], [xxxx], [xxxx], [xxxx], [xxxx]) ON [T].[xxxx] = [S].[xxxx] AND [T].[xxxx] = [S].[xxxx] AND [T].[xxxx] = [S].[xxxx] AND [T].[xxxx] = [S].[xxxx] WHEN NOT MATCHED BY TARGET THEN INSERT ([xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [Json], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx]) VALUES ([xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [xxxx], [LastUpdated], [xxxx], [xxxx], [xxxx], [xxxx]) WHEN MATCHED THEN UPDATE SET [LastUpdated] = [T].[LastUpdated], [xxxx] = [T].[Json], [xxxx] = [T].[xxxx], [xxxx] = [T].[xxxx], [xxxx] = [T].[xxxx], [xxxx] = [T].[xxxx];

From sys.sysprocesses
spid	kpid	blocked	waittype	waittime	lastwaittype	open_tran	status	program_name	cmd	stmt_start	stmt_end	request_id
151	0	0	0x0000	0	MISCELLANEOUS                   	1	sleeping                      	Core Microsoft SqlClient Data Provider                                                                                AWAITING COMMAND          	-1	-1	0
152	0	0	0x0000	0	MISCELLANEOUS                   	1	sleeping                      	Core Microsoft SqlClient Data Provider                                                                                AWAITING COMMAND          	-1	-1	0

Observations:

Not reproducible when running on Windows
Reproducible when running on Docker on Windows using Linux containers
Not reproducible when using local database
Not reproducible on 2.2
Not reproducible if not wrapped with transaction

Technical details:
EF Core version: 3.0.0
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: .NET Core 3.0
Operating system: ubuntu 16.04 x64

@alex-zyl Can you post code showing what calls are actually being made against EF Core? Also, it doesn't look like multiple queries are beiung run concurrently against the same DbContext instance, but could you confirm this?

@ajcvickers
Will try to extract the code into reproducible project tomorrow.

but could you confirm this?

Correct. DbContext is instantiated and disposed per dependency.

@ajcvickers
I extracted the code into a separate project. Please see https://github.com/alex-zyl/EFCore3OnLinuxIssue
One thing that I noticed while testing is that the connection string contained MultipleActiveResultSets=True;.
With MARS turned off, no hanging executions so far, so MARS setting might be the case.

Tested in AKS: turning off MARS resolved the issue.
Wondering what changed in 3.0 comparing to 2.2

@ajcvickers
The issue seem to be unrelated to EF Core. Should I raise it in corefx repo?

@alex-zyl Hard to be sure where it should go. Do you still think it is an issue with MARS? If so, SqlClient is probably the right place.
@smitpatel Do we still have conditional behavior based on MARS in the new query pipeline?

@ajcvickers - No.

@ajcvickers
Thank you. I went through the open issues and it turned out there are already few of them describing similar behavior.


Cluster Deleting State

add styles for clusters with deleting state

fix for https://github.com/supergiant/supergiant/issues/774

Coverage decreased (-0.3%) to 61.313% when pulling 3dc7abc34d61622dc99bcfb2e9be2f22c9571c35 on josephkane:deleting-status into 4b3c024a0a4c69f0b6d9dcdf42bb9ee7571e9734 on supergiant:master.


feat(rule-checker): updated docstrings

RATIONALE
A short description of the changes in this pull request. If the pull request is related to an open issue, mention it here.
CHANGES

update rules checker docstrings

CHECKLIST

[ ] Tests covering the new functionality have been added
[ ] Documentation has been updated OR the change is too minor to be documented

Codecov Report
Patch and project coverage have no change.

Comparison is base (99f1f82) 93.54% compared to head (f0ff0cc) 93.54%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #170   +/-   ##
=======================================
  Coverage   93.54%   93.54%
=======================================
  Files          17       17
  Lines        1364     1364
  Branches      275      238   -37
=======================================
  Hits         1276     1276
  Misses         54       54
  Partials       34       34

Files Changed
Coverage Δ

...c/flake8_custom_import_rules/core/rules_checker.py
90.67% <ø> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Customizable directory for rules files

We need support for custom directory for rules file.

This changes where are those files stored on deployer host not on target, so why not just symlink it in a proper directory. File search path is not absolute so a file doesn't need to be stored inside a role directory.

Hi,
Can we re-open this? I have the same issue.
The role is installed via ansible-galaxy and therefore not checked in to my git repo.
I have my alerting rules as static files outside of the roles directory but no way to tell this role to load them from there as the fileglob is hard coded.
My workaround is to have an extra copy task in my playbook, after importing the cloudalchemy.prometheus role.
But this also means I have to manually trigger a reload on the prometheus service too, as I can't access the handler inside the role.
The cloudalchemy Grafana role solves this in the same way as this PR:
https://github.com/cloudalchemy/ansible-grafana/blob/master/defaults/main.yml#L194
IMO this is a much nicer approach than having to get my colleagues to run ansible-galaxy and then manually symlink stuff into the role directory, or having duplicate tasks/reloads.

I think your rebase went badly and you included commits which shouldn't be included in this PR. Could you fix it?

Yes... it went badly.
I recreated the branch... if you can reopen the request it should be fine now.

I cannot since a branch was removed :(


Delete Files capability not documented

We've found, catastrophically, that Guests can delete files.
We need them to be able to upload files to a channel, but not delete files once uploaded.
I can't find this in the above chart, so it's a doc bug. I'm also finding it difficult to find out where to change that setting, which may be a feature bug. It starts with the documentation though :)
It's horrible, horrible, and a data loss and compliance issue that Guests can delete files. Please tell me there's a setting somewhere to change that, and please document it somewhere easily findable.
    == John ==

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 048930a4-fdf2-9a90-3b47-46af66db3db4
Version Independent ID: 8d6a2c84-94e9-b17e-f5c1-91191187ce4e
Content: What the guest experience is like - Microsoft Teams
Content Source: Teams/guest-experience.md
Service: msteams
GitHub Login: @LanaChin
Microsoft Alias: v-lanac

@jgwinner thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.

I already did that. Support said you can't prevent Guests from deleting files.
This is a LARGE security hole and really needs to be documented.


using the label_tw.config does not work. Tower stops after 1 sec.

Describe the bug
The nf-tower run stops succesfully after 1 sec when using the /wf_utils/labels_tower.config
example
To Reproduce
Steps to reproduce the behavior:

Add --config /wf_utils/labels_tower.config too any of the task resources_scripts
execute the script

Expected behavior
Increase memory of a process if failed with a memory error for a max of 3 tries
Additional context
Add any other context about the problem here.

Closed in #384


Password for JBD BMS

The newer JBD-BMS require a PIN or password. Even if this is specified in the configuration in the same way as for victron, access to the BMS is not possible.
Is there an option or will there be an update?
JBD BMS 4S 150A +Bluetooth

Hello, have you been able to get it to connect?

No, I did not succeed.
In the meantime, I have tried another independent HA instance and have had no success with that either.
Without a PIN, the attempt ends with a timeout. With PIN I get the error message:
11:55:39 ERROR [sampling]  XXXXXXX error (#3): [org.bluez.Error.AuthenticationFailed] Authentication Failed

I see, I thought this was for the JK BMS. I found out that this addon was able to connect without adding the pin. However, the pin is the default "1234" paring pin.


Add note of supported file types

Fixes #42
The list was achieved by intersecting supported Tika formats with what is renderable via the img tag.

LGTM


Figure out a way to deal with slow operations UX wise

From<EMAIL_ADDRESS>on June 07, 2014 02:41:03
Reproduction steps:
1. Import public keys for multiple recipients (4+).
2. Compose a message and include multiple recipients on the To line.
3. Press 'Protect this message'
What do you see?
There isn't any feedback that the user pressed the Protect button. It left me wondering if I had pressed the button, or if the extension had become unresponsive.
What do you expect instead?
Change the color of the button or display a progress bar when encrypting the message.
Original issue: http://code.google.com/p/end-to-end/issues/detail?id=41

This can be solved by using the WorkerContextImpl instead of ContextImpl in the launcher, but some UI interactions should be blocked then during crypto operations.

we can do this for ECC keys via webcrypto rather than workers now, so this should be easy now

cleaning up old bugs, please reopen if you would still like to see this fixed


No 'version' key in the manifest file for custom integration 'victorsmartkill'

Logger: homeassistant.loader
Source: loader.py:802
First occurred: 13:34:45 (1 occurrences)
Last logged: 13:34:45
No 'version' key in the manifest file for custom integration 'victorsmartkill'. This will not be allowed in a future version of Home Assistant. Please report this to the maintainer of 'victorsmartkill'

Fixed :-)
Thanks.


Confusing behavior when multiple events for a single socket are fired in a single event loop iteration

Issue representing the confusion experienced in #172 and #179.
My current understanding of the confusion is, given epoll & a socket that received a HUP and is writable, mio will first call the Handler::readable w/ ReadHint set to Hup followed by the Handler::writable event. The confusion arises when the socket is closed during the Handler::readable call, or perhaps the token associated with that token is reused. The next call to Handler::writable for the "finalized" socket is unexpected.
Part of the original API design consideration is how to unify epoll w/ kqueue. With kqueue, the readable & writable notifications are provided separately. As far as I am aware, there is no guarantee as to which order (readable or writable) the events will be seen.
I don't believe that either solutions described by @rrichardson in #179 would be possible to implement (efficiently) for kqueue. The reason why hup / error are provided as a ReadHint is that it is not 100% portable for mio to detect them. However, the hup / error state can be discovered by the mio by reading from the socket, which is why they are considered readable events. In other words, to correctly use mio, hup / error states must be discovered by listening for Handler::readable, reading from the socket, and seeing an error. As such, doing anything with the ReadHint argument passed to Handler::readable is simply an optimization. I also don't see a way to implement passing ReadHint (renamed to EventHint) to Handler::writable callback in a portable fashion.

In this case, the confusion arose when the user got a hup and then a writable event for the same token. In isolation, the obvious thing to do when you get a hup event is to unregister the socket and remove it and the token from the slab.  However, immediately, the writable event is called for the token, regardless of what one does with the token.  This results in unexpected behavior.
Given the current design, one would have to update the state in their connection's state machine indicating a disconnection, so that they can ignore the writable callback.
An approach that would allow them to not have to manage a potential disconnection in their state machine is to simply pass the ReadHint information to both readable and writable, then the convention would be to check the  Hint for err/hup before touching the socket.  (more specifically, on a read event, one would likely read then deregister, and on a write event, they would just deregister)
If they're on OSX, then they'll just have to understand that they'll never get formal hup/err hints (afaik) so nothing really changes.

The current behavior / API is also problematic when dealing with the remote end shutting down part of the connection....

Yes. That scenario applies to my above comment.
ping @dpc and @hoxnox

I couldn't live with that API and fell down to nix-rust and epoll. Fortunately my target platform is Linux. Surprisingly code became simpler, cleaner and more stable.
AS for me, if I have chosen something cross-platform, I would rather ported libevent.

I can confirm that this is confusing. I don't have much insight on a best design, but here's my view. From my perspective breaking up events into separate handler is not a best idea. It seems to me if one handler would be called for any events, with an argument what happened than it might be better. On some backends it might always be single event, on some there might be grouping.
I find current way inconsistent. Eg. Why is hangup passed as an argument to read? If there's already argument passed on what type of event happened, why not go all the way with it.
In my code for every handler called, I need to do a lookup of an object to act on. If I had the events as a bitmask, than I can act on all events grouped together, with just one lookup.
And of course, if the events are not artificially broken up, there is no problem of ghost events after deregister.

@dpc @hoxnox  The problem is that events are broken up on kqueue. This is going to require a bit of thought... I'm all for unifying the Handler callback to a ready, but I am not 100% sure how best to support both epoll & kqueue in a portable way.

@dpc Also, for what it is worth, Poll is a lower level wrapper around both epoll & kqueue that doesn't worry about normalizing events.

@carllerche libevent supports kqueue as well as epoll and, as I know, there is no problems with it. Maybe it's sources helps.

@carllerche : I don't understand. What I'm saying is that mio should not additionally break up events.
pub trait Handler {
    ...
    fn readable(&mut self, event_loop: &mut EventLoop<Self>, token: Token, hint: ReadHint) { ... }
    fn writable(&mut self, event_loop: &mut EventLoop<Self>, token: Token) { ... }
    ....
}

Could be changed to
pub trait Handler {
    ...
    fn event(&mut self, event_loop: &mut EventLoop<Self>, token: Token, event: EventBitmask) { ... }
    ....
}

EventBitmask would contain bits for "readable, writeable, hup" and maybe others.
Now if the backend is always reporting one event at the time, than only one bit will be set. If backend supports reporting more events in the same event, then more bits will be set. Handler code will handle both cases just fine.
Now the handler will go through events and check them in order it wants.

I don't entirely disagree here. There is a number (technically infinite) of
event types that a poller could return, dividing into only read/write is
somewhat arbitrary.
On Sun, May 31, 2015, 5:02 PM Dawid Ciężarkiewicz<EMAIL_ADDRESS>wrote:

@carllerche https://github.com/carllerche : I don't understand. What
I'm saying is that mio should not additionally break up events.
pub trait Handler {
...
fn readable(&mut self, event_loop: &mut EventLoop, token: Token, hint: ReadHint) { ... }
fn writable(&mut self, event_loop: &mut EventLoop, token: Token) { ... }
....
}
Could be changed to
pub trait Handler {
...
fn event(&mut self, event_loop: &mut EventLoop, token: Token, event: EventBitmask) { ... }
....
}
EventBitmask would contain bits for "readable, writeable, hup" and maybe
others.
Now if the backend is always reporting one event at the time, than only
one bit will be set. If backend supports reporting more events in the same
event, then more bits will be set. Handler code will handle both cases just
fine.
Now the handler will go through events and check them in order it wants.
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-107246101.

I do run into this issue in my reactor framework. I have to store the fact
that I got a failed connection from my read handler knowing that the write
handler is about to be spuriously called.
On Sun, May 31, 2015, 5:06 PM Rick Richardson<EMAIL_ADDRESS>wrote:

I don't entirely disagree here. There is a number (technically infinite)
of event types that a poller could return, dividing into only read/write is
somewhat arbitrary.
On Sun, May 31, 2015, 5:02 PM Dawid Ciężarkiewicz <
<EMAIL_ADDRESS>wrote:

@carllerche https://github.com/carllerche : I don't understand. What
I'm saying is that mio should not additionally break up events.
pub trait Handler {
...
fn readable(&mut self, event_loop: &mut EventLoop, token: Token, hint: ReadHint) { ... }
fn writable(&mut self, event_loop: &mut EventLoop, token: Token) { ... }
....
}
Could be changed to
pub trait Handler {
...
fn event(&mut self, event_loop: &mut EventLoop, token: Token, event: EventBitmask) { ... }
....
}
EventBitmask would contain bits for "readable, writeable, hup" and maybe
others.
Now if the backend is always reporting one event at the time, than only
one bit will be set. If backend supports reporting more events in the same
event, then more bits will be set. Handler code will handle both cases just
fine.
Now the handler will go through events and check them in order it wants.
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-107246101.

@dpc I'm not against this, but could you elaborate on how this would behave with kqueue?

@carllerche : Unfortunately my kqueue knowledge is null. What is the problem there?

The problem is that events are broken up on kqueue.

The way I understand it, kqueue signals single event every time, and EventBitmask from my example will always have only one bit set?

@dpc Kqueue treats readable & writable events on a socket as completely separate (unlike epoll). See here.
Basically, the current mio design of having separate events for readable and writable was made to handle kqueue while adding as little overhead as possible. The strategy to implement an event function that coalesces both readable & writable events that works on kqueue does not seem obvious to me at first thought.

I see two options:

Coalesce the event masks into a single mask per file-descriptor and then
invoke the event() callback.
Just call the event() callback multiple times, one for each event.

Since kqueue and others can produce user defined events,  I am wondering if
there  is a way to extend the amount/type of events event_loop can produce.
On Mon, Jun 1, 2015 at 1:39 PM Carl Lerche<EMAIL_ADDRESS>wrote:

@dpc https://github.com/dpc Kqueue treats readable & writable events on
a socket as completely separate (unlike epoll). See here
https://github.com/carllerche/mio/blob/master/src/sys/unix/kqueue.rs#L39-L40
.
Basically, the current mio design of having separate events for readable
and writable was made to handle kqueue while adding as little overhead as
possible. The strategy to implement an event function that coalesces both
readable & writable events that works on kqueue does not seem obvious to me
at first thought.
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-107650533.

@rrichardson how would you implement option 1? The best that I can think of is to either use a hash lookup structure to coalesce the events each iteration or use a Slab to map every token registered w/ EventLoop with the event masks.

I can't think of a way to do it other than some sort of associative map.
It would probably be optimal to use a local radix tree that is sized
directly to the amount of fds that kevent() can return at one time.
create map;
iterate through kevent array, inserting and updating map;
iterate through map invoking callbacks;
On Mon, Jun 1, 2015 at 2:49 PM Carl Lerche<EMAIL_ADDRESS>wrote:

@rrichardson https://github.com/rrichardson how would you implement
option 1? The best that I can think of is to either use a hash lookup
structure to coalesce the events each iteration or use a Slab to map
every token registered w/ EventLoop with the event masks.
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-107669325.

Sorry, I still don't understand where is the problem, @carllerche . In my example on both readable and writeable you call the same event with argument bitmask having different bit set. There is no need to coalesce anything artificially, if the backend did not report it already together. What am I missing?

@dpc Doing this would mean that the original confusing behavior would still be around, but only on some platforms making code that is written w/ mio still have to factor it in but also have to handle differences between platforms.
Currently, if you want to drop down to a lower level and handle platform differences yourself, you can use the Poll type which will just provide you with the events directly.

Why, because underlying backend can report additional events on it's own?

kqueue will produce N event structures for a single file descriptor in a single call of kevent. Those events are intermixed with all of the events for all of the other fds it is monitoring.  So if you want a single callback per descriptor that is effected, then we absolutely need to coalesce the events into a single bitmask.
epoll is easier, to be sure, it coalesces the events for us, so we only get one event structure per fd after polling.
My suggestion is that we fix the "bug" in the interrim by not invoking readable for HUP/ERR, but rather pass ReadHint to both readable and writable events when found. This should work for both epoll and kqueue.
My understanding of kqueue and EV_EOF is sketchy, but it sounds like kqueue users aren't used to getting hup or error notifications anyways, they just get those from the socket/fd.

@rrichardson Would a radix tree be better than some sort of simple hashing? Right now, kqueue is configured to pull in at most 1024 events. Given that, it seems that a 4k vec could be allocated and some very simple hashing strategy used... (very cheap hash function & robin hood hash collision strategy)
The hashing algorithm could be as simple as masking the lower 12 bits of the token. Since tokens are usually taken from Slab, and Slab uses numbers starting at 0 and increments (and reuses slots that are cleared), in most simple cases there should be no collisions... in most cases, it would take more than 4k live connections to start hitting collisions, in which case there would be some linear probing.
The size of the original vec could also be configured, so if using kqueue w/ very large numbers of live connections, the vec size could be increased to reduce collisions.

Recording some thoughts from IRC:
An alternative approach to solving this is to instead make deregistering an asynchronous operation, so the user is on the hook for handling events on deregistered tokens/fds until that token is confirmed deregistered.
There are several ways to accomplish this. The ideal interface is to add another method on Handler, lets say deregistered, which gets called with a Token once it is deregistered and no more events are pending on it. However, this requires some slight overhead within mio to track deregistrations, which is not ideal.
A lower-implied-overhead approach is to instead add another method to Handler, lets say tick, which is called every time the event loop has entirely finished dispatching events from the current iteration before the next call into epollorkqueue. Deregistrations "take effect" on every call to tick, which would then be the place for users to act on those deregistrations. This approach practically require users to maintain a list of deregistrations between tick` calls, but it is much less opinionated than doing the same within mio.

Note that adding tick or deregistered is orthogonal to collapsing readable and writable into one method, which could be done or not either way.

The tick imposes on the user the same tracking that mio would have to do internally for deregistered case, doesn't it?

Yea, but it would allow different implementations depending on the concerns of the user. This is similar to the overhead "imposed" by mio due to the token system.
Thanks,
Jonathan Reem
On Tue, Jun 2, 2015 at 7:20 PM, Dawid Ciężarkiewicz
<EMAIL_ADDRESS>wrote:

The tick imposes on the user the same tracking that mio would have to do internally for deregistered case, doesn't it?
Reply to this email directly or view it on GitHub:
https://github.com/carllerche/mio/issues/184#issuecomment-108163571

Having a final "no more events for this iteration" achieves the same effect as ensuring that all events are sent in a single event(), both have their pros and cons. The final handler would be easier to implement in the general case, the single-handler approach is certainly simpler from an a trait interface perspective.  Also, I do want to point out that in either of these cases, an async deregister aren't really necessary. It is safe in Mio to deregister in an event callback.
The async deregister came up because we were discussing the specific case in libuv and higher level mio frameworks where the callback handler is registered with the token, so it takes some special gymnastics to manage.

While it may be safe in Mio to de-register a token in the midst of handling events, client code still needs to know when it's safe to release data structures associated with the token. Otherwise, the token exists in a sort of gray area, not dissimilar to a libuv handle being stuck with UV_CLOSING set, without a state transition to UV_CLOSED.

Definitely, but if event() is only fired a max of one time per iteration, then we know that callback is final.   The owner of the data structures doesn't need a state machine in that case.
Frankly, if we can get some nods in either direction I'm happy to put in a PR for either approach.  Or we can ask the benevolent dictator :P

What about the case of, say, token1 de-registering token2 before token2 has had a chance to run?

When a poll() waits and completes is effectively arbitrary, but it probably does so more than a thousand times a second on a busy system. Who says that token2 will even get a chance to run during that iteration?
If token2 needs to complete some critical operation before closing down, then it needs to enforce that invariant, not leave it up to potentially racy logic.

That said, I guess it's not a huge amount of overhead to have such a final() callback, who am I to judge someone's kinks. If that's what they're into, we can supply.
OK.
My proposal:

remove readable and writeable, change to poll_event()  (I think event() is too generic, since there are still other handlers, notify and timeout)
Coalesce all events into a single mask per file descriptor for every poll() run.
Add a final() function.  Maybe pass in the number of events handled on this run (or some other interesting statistics for reporting)

@rrichardson I'm fine w/ that proposal. The specific coalescing strategy doesn't have to be optimal yet since I believe the only officially supported kqueue system as of today is OS X. I am confident enough that it is possible to implement a very low overhead coalescing strategy in the future. It would be nice if the implementation didn't allocate though.
Now, bike shedding wise... what about ready instead of poll_event()?

I like ready().
On Wed, Jun 3, 2015, 1:07 PM Carl Lerche<EMAIL_ADDRESS>wrote:

@rrichardson https://github.com/rrichardson I'm fine w/ that proposal.
The specific coalescing strategy doesn't have to be optimal yet since I
believe the only officially supported kqueue system as of today is OS X. I
am confident enough that it is possible to implement a very low overhead
coalescing strategy in the future. It would be nice if the implementation
didn't allocate though.
Now, bike shedding wise... what about ready instead of poll_event()?
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-108526131.

If we have 3 in @rrichardson's proposal, do we actually need to do 2? That decision could be left to user code, so we don't imply the overhead of coalescing on kqueue.

2 is mostly for ergonomics and the principle of last surprise. As long as
we pass hints to each ready() call, there is no actual need. It just makes
EventLoop behave the same regardless of whether it's over kevent or epoll.
Otherwise the use customer code has to be built to anticipate both models.
On Wed, Jun 3, 2015, 3:17 PM Jonathan Reem<EMAIL_ADDRESS>wrote:

If we have 3 in @rrichardson https://github.com/rrichardson's proposal,
do we actually need to do 2? That decision could be left to user code, so
we don't imply the overhead of coalescing on kqueue.
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-108585649.

That should read principle of least surprise grumble mobile

Also, I think the strategy that I outlined above should be minimal overhead and could be tuned to be close to 0

If there's no noticeable performance cost to 2 then I am for it, it certainly does make the programming model simpler.

@reem the strategy would be what was outlined here. Thoughts?

@carllerche the std HashMap allows you to define your own hasher and hashing algorithm for your key type and has a very optimized robin hood implementation and allocation strategy, so we could probably use that rather than adding complexity to mio if there is some way to prevent it from resizing. Other than that it sounds fine, hard to say more without looking at real perf numbers.

Yea, being able to preallocate slots sized to a safe value for the max
number of events returned by kqueue is also very helpful.  It would be nice
to to get insight into the stats of the hashmap to know how effective the
sizing is in terms of rehashing etc
On Wed, Jun 3, 2015, 5:35 PM Jonathan Reem<EMAIL_ADDRESS>wrote:

@carllerche https://github.com/carllerche the std HashMap allows you to
define your own hasher and hashing algorithm for your key type and has a
very optimized robin hood implementation and allocation strategy, so we
could probably use that rather than adding complexity to mio if there is
some way to prevent it from resizing. Other than that it sounds fine, hard
to say more without looking at real perf numbers.
—
Reply to this email directly or view it on GitHub
https://github.com/carllerche/mio/issues/184#issuecomment-108622822.

added #194  to address this.

An initial pass of this is almost ready to merge: https://github.com/carllerche/mio/tree/coalesce-events

Thank you so much @carllerche!!!
We should probably use a faster hash in the HashMap though.

@arthurprs , relevant: https://users.rust-lang.org/t/fastest-lookup-time-map-for-short-keys/2028

For keys <= 8 bytes it's really hard to beat FNV as it's super simple and small (thus inlined).

@arthurprs This is definitely an initial pass implementation. I'm thinking that I will probably just use a mask as the hash function. Given that tokens tend to be sequential (when used with a slab), this should have a fairly good collision rate. Also, allocating the map capacity to the max # of connections will completely avoid collisions at the expense of memory.

Yeah, with the robin-hood under the good we can probably get away just returning the integer plus (a bit of overallocation won't hurt either).
Heck python gets away with it (hashing integers as themselves) using a more basic open addressing and 67% resize policy.


Skip list

CODE REVIEW
This is a basic implementation of a skip list with add/delete/get/iterate functionality.  I think there are some optimizations that can be made here, and I believe I can combine some aspects of this with some concepts from the y-fast trie to generate a new type of datastructure with excellent cache locality.
Benchmarks:
BenchmarkInsert-8	 1000000	      1104 ns/op
BenchmarkGet-8	 3000000	       547 ns/op
BenchmarkDelete-8	 3000000	       483 ns/op
These numbers are not deterministic due to the nature of how a skip list works, so they may fluctuate between runs.
@alexandercampbell-wf @beaulyddon-wf @tannermiller-wf @rosshendrickson-wf @ericolson-wf @stevenosborne-wf @tylertreat-wf

+1

+1

+1

+1


Using volume to map SSL certificates can cause certificate mismatch

Make sure you read and understand http://emqtt.io/docs/v2/index.html.
Use one of the two templates below and delete the rest.

[x] BUG REPORT

BUG REPORT

For detailed steps, please check out the Test code below. My question is whether this problem is caused by my incorrect configuration or it's a bug in emqx. I tested all versions of 3.x and the issue exists.
BTW, I want to ask what this means?
[Alarm Handler] New Alarm: cpu_high_watermark, Alarm Info: 100.0

Expected behavior
Can be successfully connected via SSL.
Actual behavior
emqx reported that the certificate does not match.
Test code
 First attempt (Failed)
I have an instance of emqx running in the docker container, configured as follows:
emqx:
    container_name: my_emqx
    image: emqx/emqx:v3.2.1
    restart: always
    ports:
      - "1883:1883"
      - "8883:8883"
      - "18083:18083"
      - "18084:18084"
    volumes:
      - $PWD/config/emqx/certs:/opt/emqx/etc/certs:rw
      - $PWD/config/emqx/add-user.sh:/opt/emqx/add-user.sh:ro
    networks:
      - frontend
      - backend
    environment:
      - TZ=Asia/Shanghai
      - EMQX_LOADED_PLUGINS="emqx_management emqx_recon emqx_retainer emqx_dashboard emqx_auth_username"
      - EMQX_LOG__LEVEL=debug
      - EMQX_ALLOW_ANONYMOUS=false
      - EMQX_ACL_NOMATCH=deny
      - EMQX_MQTT__MAX_PACKET_SIZE=10MB
      - EMQX_LISTENER__SSL__EXTERNAL__TLS_VERSIONS=tlsv1.2,tlsv1.1,tlsv1
      - EMQX_LISTENER__SSL__EXTERNAL__DHFILE=etc/certs/dh-params.pem
      - EMQX_LISTENER__SSL__EXTERNAL__VERIFY=verify_peer
      - EMQX_LISTENER__SSL__EXTERNAL__FAIL_IF_NO_PEER_CERT=true
      - EMQX_LISTENER__SSL__EXTERNAL__REUSE_SESSIONS=on
      - EMQX_LISTENER__SSL__EXTERNAL__KEYFILE=etc/certs/server.key
      - EMQX_LISTENER__SSL__EXTERNAL__CERTFILE=etc/certs/server.pem
      - EMQX_LISTENER__SSL__EXTERNAL__CACERTFILE=etc/certs/ca.pem

Here is the log:
[vagrant@localhost docker]$ docker-compose up --build emqx
Creating network "docker_backend" with the default driver
Creating network "docker_frontend" with the default driver
Creating my_emqx ... done
Attaching to my_emqx
my_emqx  | listener.ssl.external.dhfile=etc/certs/dh-params.pem
my_emqx  | node.max_ports=1048576
my_emqx  | listener.ssl.external.verify=verify_peer
my_emqx  | listener.tcp.external.acceptors=64
my_emqx  | log.level=debug
my_emqx  | mqtt.max_packet_size=10MB
my_emqx  | listener.ssl.external.certfile=etc/certs/server.pem
my_emqx  | listener.ssl.external.fail_if_no_peer_cert=true
my_emqx  | allow_anonymous=false
my_emqx  | listener.ssl.external.acceptors=32
my_emqx  | listener.ssl.external.keyfile=etc/certs/server.key
my_emqx  | node.process_limit=2097152
my_emqx  | node.max_ets_tables=2097152
my_emqx  | listener.ws.external.acceptors=16
my_emqx  | listener.ssl.external.cacertfile=etc/certs/ca.pem
my_emqx  | listener.ssl.external.reuse_sessions=on
my_emqx  |<EMAIL_ADDRESS>my_emqx  | listener.ssl.external.tls_versions=tlsv1.2,tlsv1.1,tlsv1
my_emqx  | acl_nomatch=deny
my_emqx  | EMQX_LOADED_PLUGINS="emqx_management emqx_recon emqx_retainer emqx_dashboard emqx_auth_username"
my_emqx  | emqx v3.2.1 is started successfully!
my_emqx  |               {mfargs,{prometheus_histogram,start_link,[]}},
my_emqx  |               {restart_type,permanent},
my_emqx  |               {shutdown,5000},
my_emqx  |               {child_type,worker}]
my_emqx  | 2019-07-31 15:48:34.675 [info] application: prometheus
my_emqx  |     started_at<EMAIL_ADDRESS>my_emqx  | 2019-07-31 15:48:34.682 [info] application: luerl
my_emqx  |     started_at<EMAIL_ADDRESS>my_emqx  | 2019-07-31 15:48:34.682 [info] application: xmerl
my_emqx  |     started_at<EMAIL_ADDRESS>my_emqx  | ['2019-07-31T15:48:40Z']:emqx start

As you can see that it has been successfully run. At this point I enter the container and add a few users. Because you changed this mechanism, it is a little troublesome.
[vagrant@localhost ~]$ docker exec -it my_emqx sh
/opt/emqx $ sh add-user.sh
init terminating in do_boot ({terminated,[{io,format,[<0.63.0>,[_],[_]],[]},{escript,start,1,[{_},{_}]},{init,start_em,1,[]},{init,do_boot,3,[]}]})

Crash dump is being written to: erl_crash.dump...done
ok
ok
ok
ok
/opt/emqx $

The contents of the add-user.sh script are as follows (sensitive information has been modified):
#!/bin/sh

set -e

ctl_cmd=/opt/emqx/bin/emqx_ctl

addUser() {
	$ctl_cmd users add u1 p1
	$ctl_cmd users add u2 p2
	$ctl_cmd users add u3 p3
	$ctl_cmd users add u4 p4
}

if $ctl_cmd users | grep -q "users list"
then
	addUser
else
	$ctl_cmd plugins load emqx_auth_username
	addUser
fi

At this point emqx output log is as follows:
my_emqx  | 2019-07-31 15:49:12.650 [error] [Ctl] CMD Error:terminated, Stacktrace:[{io,format,
my_emqx  |                                          [<41476.63.0>,"~-48s# ~s~n",
my_emqx  |                                           ["users update <Username> <NewPassword>",
my_emqx  |                                            "Update User"]],
my_emqx  |                                          []},
my_emqx  |                                         {emqx_cli,'-usage/1-fun-0-',1,
my_emqx  |                                          [{file,
my_emqx  |                                            "/emqx_rel/_build/emqx/lib/emqx/src/emqx_cli.erl"},
my_emqx  |                                           {line,32}]},
my_emqx  |                                         {lists,map,2,
my_emqx  |                                          [{file,"lists.erl"},{line,1239}]},
my_emqx  |                                         {lists,map,2,
my_emqx  |                                          [{file,"lists.erl"},{line,1239}]},
my_emqx  |                                         {emqx_ctl,run_command,2,
my_emqx  |                                          [{file,
my_emqx  |                                            "/emqx_rel/_build/emqx/lib/emqx/src/emqx_ctl.erl"},
my_emqx  |                                           {line,80}]},
my_emqx  |                                         {rpc,'-handle_call_call/6-fun-0-',5,
my_emqx  |                                          [{file,"rpc.erl"},{line,197}]}]

I don't know what it means, so I choose to ignore it because it works fine.
At this point I try to connect to the instance, I use MQTT.fx as the test client, the connection is successful, the emqx log is as follows:
my_emqx  | 2019-07-31 15:52:15.187 [debug] <IP_ADDRESS>:60639 [Channel] RECV <<16,67,0,4,77,81,84,84,4,194,0,60,0,14,77,81,84,84,95,70,88,95,
my_emqx  |                  67,108,105,101,110,116,0,21,114,111,121,98,105,45,109,113,116,
my_emqx  |                  116,64,110,111,100,101,115,101,114,118,101,114,0,16,120,56,77,
my_emqx  |                  80,69,107,81,86,97,53,70,55,51,115,114,81>>
my_emqx  | 2019-07-31 15:52:15.188 [debug] <IP_ADDRESS>:60639 [Protocol] RECV CONNECT(Q0, R0, D0, ClientId=MQTT_FX_Client, ProtoName=MQTT, ProtoVsn=4, CleanStart=true, KeepAlive=60, Username=u1, Password=******)
my_emqx  | 2019-07-31 15:52:15.188 [debug] MQTT_FX_Client@<IP_ADDRESS>:60639 [Protocol] SEND CONNACK(Q0, R0, D0, AckFlags=0, ReasonCode=0)

Then I disconnected, the emqx log is as follows:
my_emqx  | 2019-07-31 15:54:08.568 [debug] MQTT_FX_Client@<IP_ADDRESS>:60639 [Channel] RECV <<224,0>>
my_emqx  | 2019-07-31 15:54:08.568 [debug] MQTT_FX_Client@<IP_ADDRESS>:60639 [Protocol] RECV DISCONNECT(Q0, R0, D0, ReasonCode=0)
my_emqx  | 2019-07-31 15:54:08.568 [debug] MQTT_FX_Client@<IP_ADDRESS>:60639 [Channel] Terminated for normal
my_emqx  | 2019-07-31 15:54:08.569 [info] MQTT_FX_Client@<IP_ADDRESS>:60639 [Protocol] Shutdown for normal

At this point I tried to use the SSL connection, the connection failed, I guarantee that the client configuration is correct, the emqx log is as follows:
my_emqx  | 2019-07-31 15:54:49.943 [error] crasher:
my_emqx  |     initial call: emqx_channel:init/1
my_emqx  |     pid: <0.2148.0>
my_emqx  |     registered_name: []
my_emqx  |     exception error: no match of right hand side value
my_emqx  |                      {error,
my_emqx  |                          {ssl_error,
my_emqx  |                              {options,
my_emqx  |                                  {cacertfile,"etc/certs/ca.pem",
my_emqx  |                                      {error,efault}}}}}
my_emqx  |       in function  emqx_channel:init/1 (/emqx_rel/_build/emqx/lib/emqx/src/emqx_channel.erl, line 152)
my_emqx  |     ancestors: [<0.1719.0>,<0.1718.0>,esockd_sup,<0.1354.0>]
my_emqx  |     message_queue_len: 0
my_emqx  |     messages: []
my_emqx  |     links: [<0.1719.0>]
my_emqx  |     dictionary: []
my_emqx  |     trap_exit: true
my_emqx  |     status: running
my_emqx  |     heap_size: 4185
my_emqx  |     stack_size: 27
my_emqx  |     reductions: 5426
my_emqx  |   neighbours:
my_emqx  | 2019-07-31 15:54:49.943 [error] supervisor: 'esockd_connection_sup - <0.1719.0>'
my_emqx  |     errorContext: connection_crashed
my_emqx  |     reason: {{badmatch,
my_emqx  |                  {error,
my_emqx  |                      {ssl_error,
my_emqx  |                          {options,
my_emqx  |                              {cacertfile,"etc/certs/ca.pem",
my_emqx  |                                  {error,efault}}}}}},
my_emqx  |              [{emqx_channel,init,1,
my_emqx  |                   [{file,
my_emqx  |                        "/emqx_rel/_build/emqx/lib/emqx/src/emqx_channel.erl"},
my_emqx  |                    {line,152}]},
my_emqx  |               {proc_lib,init_p_do_apply,3,
my_emqx  |                   [{file,"proc_lib.erl"},{line,249}]}]}
my_emqx  |     offender: [{pid,<0.2148.0>},
my_emqx  |                {name,connection},
my_emqx  |                {mfargs,{emqx_channel,start_link,
my_emqx  |                                      [[{deflate_options,[]},
my_emqx  |                                        {max_conn_rate,500},
my_emqx  |                                        {active_n,100},
my_emqx  |                                        {zone,external}]]}}]
my_emqx  | 2019-07-31 15:54:49.946 [error] crasher:
my_emqx  |     initial call: emqx_channel:init/1
my_emqx  |     pid: <0.2152.0>
my_emqx  |     registered_name: []
my_emqx  |     exception error: no match of right hand side value
my_emqx  |                      {error,
my_emqx  |                          {ssl_error,
my_emqx  |                              {options,
my_emqx  |                                  {cacertfile,"etc/certs/ca.pem",
my_emqx  |                                      {error,efault}}}}}
my_emqx  |       in function  emqx_channel:init/1 (/emqx_rel/_build/emqx/lib/emqx/src/emqx_channel.erl, line 152)
my_emqx  |     ancestors: [<0.1719.0>,<0.1718.0>,esockd_sup,<0.1354.0>]
my_emqx  |     message_queue_len: 0
my_emqx  |     messages: []
my_emqx  |     links: [<0.1719.0>]
my_emqx  |     dictionary: []
my_emqx  |     trap_exit: true
my_emqx  |     status: running
my_emqx  |     heap_size: 4185
my_emqx  |     stack_size: 27
my_emqx  |     reductions: 5413
my_emqx  |   neighbours:
my_emqx  | 2019-07-31 15:54:49.946 [error] supervisor: 'esockd_connection_sup - <0.1719.0>'
my_emqx  |     errorContext: connection_crashed
my_emqx  |     reason: {{badmatch,
my_emqx  |                  {error,
my_emqx  |                      {ssl_error,
my_emqx  |                          {options,
my_emqx  |                              {cacertfile,"etc/certs/ca.pem",
my_emqx  |                                  {error,efault}}}}}},
my_emqx  |              [{emqx_channel,init,1,
my_emqx  |                   [{file,
my_emqx  |                        "/emqx_rel/_build/emqx/lib/emqx/src/emqx_channel.erl"},
my_emqx  |                    {line,152}]},
my_emqx  |               {proc_lib,init_p_do_apply,3,
my_emqx  |                   [{file,"proc_lib.erl"},{line,249}]}]}
my_emqx  |     offender: [{pid,<0.2152.0>},
my_emqx  |                {name,connection},
my_emqx  |                {mfargs,{emqx_channel,start_link,
my_emqx  |                                      [[{deflate_options,[]},
my_emqx  |                                        {max_conn_rate,500},
my_emqx  |                                        {active_n,100},
my_emqx  |                                        {zone,external}]]}}]

 Second attempt (Successful)
Based on the configuration of the first attempt, I removed the following from volume:
- $PWD/config/emqx/certs:/opt/emqx/etc/certs:rw

Then execute the following command:
[vagrant@localhost docker]$ docker-compose down -v
Removing my_emqx ... done
Removing network docker_backend
Removing network docker_frontend

[vagrant@localhost docker]$ docker-compose up --build emqx
Creating network "docker_backend" with the default driver
Creating network "docker_frontend" with the default driver
Creating my_emqx ... done
Attaching to my_emqx
my_emqx  | listener.ssl.external.dhfile=etc/certs/dh-params.pem
my_emqx  | node.max_ports=1048576
my_emqx  | listener.ssl.external.verify=verify_peer
my_emqx  | listener.tcp.external.acceptors=64
my_emqx  | log.level=debug
my_emqx  | mqtt.max_packet_size=10MB
my_emqx  | listener.ssl.external.certfile=etc/certs/server.pem
my_emqx  | listener.ssl.external.fail_if_no_peer_cert=true
my_emqx  | allow_anonymous=false
my_emqx  | listener.ssl.external.acceptors=32
my_emqx  | listener.ssl.external.keyfile=etc/certs/server.key
my_emqx  | node.process_limit=2097152
my_emqx  | node.max_ets_tables=2097152
my_emqx  | listener.ws.external.acceptors=16
my_emqx  | listener.ssl.external.cacertfile=etc/certs/ca.pem
my_emqx  | listener.ssl.external.reuse_sessions=on
my_emqx  |<EMAIL_ADDRESS>my_emqx  | listener.ssl.external.tls_versions=tlsv1.2,tlsv1.1,tlsv1
my_emqx  | acl_nomatch=deny
my_emqx  | EMQX_LOADED_PLUGINS="emqx_management emqx_recon emqx_retainer emqx_dashboard emqx_auth_username"
my_emqx  | emqx v3.2.1 is started successfully!
my_emqx  |               {mfargs,{prometheus_histogram,start_link,[]}},
my_emqx  |               {restart_type,permanent},
my_emqx  |               {shutdown,5000},
my_emqx  |               {child_type,worker}]
my_emqx  | 2019-07-31 16:22:34.837 [info] application: prometheus
my_emqx  |     started_at<EMAIL_ADDRESS>my_emqx  | 2019-07-31 16:22:34.843 [info] application: luerl
my_emqx  |     started_at<EMAIL_ADDRESS>my_emqx  | 2019-07-31 16:22:34.843 [info] application: xmerl
my_emqx  |     started_at<EMAIL_ADDRESS>my_emqx  | ['2019-07-31T16:22:40Z']:emqx start

I repeat the first attempted steps until I am ready to connect via SSL. As of now, everything is the same as the first step.
Now I run the following command in another session window to manually copy the SSL certificates into the container.
docker cp  /usr/local/src/my-project/docker/config/emqx/certs my_emqx:/opt/emqx/etc/

Now I connect to the emqx instance via SSL. The connection is successful, here is the log
my_emqx  | 2019-07-31 16:25:42.195 [debug] <IP_ADDRESS>:61869 [Channel] RECV <<16,67,0,4,77,81,84,84,4,194,0,60,0,14,77,81,84,84,95,70,88,95,
my_emqx  |                  67,108,105,101,110,116,0,21,114,111,121,98,105,45,109,113,116,
my_emqx  |                  116,64,110,111,100,101,115,101,114,118,101,114,0,16,120,56,77,
my_emqx  |                  80,69,107,81,86,97,53,70,55,51,115,114,81>>
my_emqx  | 2019-07-31 16:25:42.195 [debug] <IP_ADDRESS>:61869 [Protocol] RECV CONNECT(Q0, R0, D0, ClientId=MQTT_FX_Client, ProtoName=MQTT, ProtoVsn=4, CleanStart=true, KeepAlive=60, Username=u1, Password=******)
my_emqx  | 2019-07-31 16:25:42.196 [debug] MQTT_FX_Client@<IP_ADDRESS>:61869 [Protocol] SEND CONNACK(Q0, R0, D0, AckFlags=0, ReasonCode=0)

EMQ version
v3.2.1
Docker version
Which docker-engine version?
Docker version 19.03.1, build 74b1e89
How docker info?
Client:
 Debug Mode: false

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 49
 Server Version: 19.03.1
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-957.5.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 1.795GiB
 Name: localhost.localdomain
 ID: JGSM:QO5H:TDQV:QSLA:UZCT:DZH4:GCBC:QWR5:TB5S:S5KU:NPJE:66ZR
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http://<IP_ADDRESS>:8888
 HTTPS Proxy: http://<IP_ADDRESS>:8888
 No Proxy: localhost,<IP_ADDRESS>
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  <IP_ADDRESS>/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

System
What system do you use?
[vagrant@localhost ~]$ cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
[vagrant@localhost ~]$ cat /proc/version
Linux version 3.10.0-957.5.1.el7.x86_64<EMAIL_ADDRESS>(gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Fri Feb 1 14:54:57 UTC 2019

Hardware
How is the host machine?
The host is a vagrant virtual machine created specifically to test the problem, running on VirtualBox with dual-core CPU and 2 GB of memory. There are no other programs running. This problem also exists on my production server on AWS. I can't I did the test on the production server, so I deployed the test environment. I think the problem has nothing to do with the host environment, so I didn't attach the Vagrantfile, but I can attach it if you need it.

Thank you for your attention. I need to point out that the owner of all files in the etc directory is emqx:emqx, the permissions of the directory where the host mapping file is located are not root.

@VincentSit Sorry for the late reply. First, the error of add_user is our problem, and we will fix it later. Second, regarding the SSH connection error, can you show us your client configuration?

@zhanghongtong Thank you for your response, here is the configuration.

@VincentSit We are trying to reproduce your issue. I have a question about docker：

I repeat the first attempted steps until I am ready to connect via SSL. As of now, everything is the same as the first step.
Now I run the following command in another session window to manually copy the SSL certificates into the container.
docker cp  /usr/local/src/my-project/docker/config/emqx/certs my_emqx:/opt/emqx/etc/

Did you restart EMQX after you executed docker cp ?

@zhanghongtong Sorry for the late reply. I didn't restart the EMQX, all operations including logs are as above.

@VincentSit I'm sorry we didn't reproduce the problem. We create the container from docker-compose.yaml. It's all right
Anything missing?

I'm not too sure either. I'll pack the whole test environment for you. But it may be a little late to have time to do it. Thank you!

The complete step is like above, And I did a test before opening the issue to make sure it could reproduce the problem. I don't know why you can't reproduce it, please wait for me to pack a whole test environment for you.


JMModal and JMAlert not recognized

Was working fine for months are suddenly likely due to update, wouldn t compile anymore.
view has no type JMAlert nore JMModal
I tried to uninstall and reinstall but no difference . CLean buil , nodifference. Very similar to a closed issue:
who had those elements
XCode version 12.4
MacOS Catalina 10.15.7
simulator model iPhone 12 pro (although doesn't even open up simulator because it fails before this
deployment version iOS 14.4
i have XCode version 12.4
MacOS Catalina 10.15.7
simulator model iPhone 11 pro
deployment version iOS 14.4
I would like to downgrade but i don t know how to retrieve myprevious version from github Any help to retrieve 1,4 would be welcome

@Itlink75 Hi there! Thanks for the issue report.
Recommended Fix
If your view has no type JMAlert nor JMModal, it's likely that Xcode did not fetch, link, import or compile the PermissionsSwiftUI package properly.
After v1.5.0, you might want to reinstall PermissionsSwiftUI and select only the submodules for permissions you use.
Please make sure that these setups are proper

Click on the blue project configuration file in the file inspector
Click on the iOS target (if you have multiple)
Verify that in the Frameworks, libraries, and embedded content section, you have the proper libraries imported.
You either need 1 submodule called PermissionsSwiftUI, OR (recommended) 1 or more submodules for permissions types you need to use.
You should NOT have the CorePermissionsSwiftUI module imported.

Verify that in the code file where JMModal or JMAlert is referenced, you are importing multiple permission submodules in the file header.

///Example.swift
import PermissionsSwiftUINotifications
import PermissionsSwiftUILocation
import PermissionsSwiftUITracking

...

var body: some View {
      .JMModal()
}

Downgrade to v1.4.0
If you want to downgrade to a previous version that works, you may do so. In Xcode, click on Add Package Dependency, click next, and it will prompt you to choose package options.

Here, you will select exact version and put in your desired version of PermissionsSwiftUI (eg. 1.4.0).

Hello,
with auto update , it wouldn t work . By reimporting main module 1.5 it would not work also .
Reimporting each module and not the main one makes the app compile again . Thanx for the support !


docs: update contributing guide for local setup

The contributing lacked instructions how to build connaisseur locally for testing.

Codecov Report

Merging #39 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #39   +/-   ##
=======================================
  Coverage   93.06%   93.06%
=======================================
  Files          14       14
  Lines         620      620
=======================================
  Hits          577      577
  Misses         43       43

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 721bd71...6d882d6. Read the comment docs.


usage of oneOf for schema validation

I am having trouble getting my schemas to validate when using oneOf structures. (running asdf=2.6.0, python=3.7)
Consider the following python/asdf example that I expected to work but somehow cannot implement.
#python class for debugging
@dataclass
class OneOfClass:
    data: dict

class OneOfClassType(CustomType):
    name = "debug/oneOf_class"
    version = "1.0.0"
    types = [OneOfClass]

    @classmethod
    def to_tree(cls, node: OneOfClass, ctx):
        return node.data

    @classmethod
    def from_tree(cls, tree, ctx):
        obj = OneOfClass(tree)
        return obj

The schema I wanted to implement:
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "http://somewhere.com/schemas/debug/oneOf_class-1.0.0"
tag: "tag:somewhere.com:debug/oneOf_class-1.0.0"

title: |
  oneOf test schema
oneOf:
  - type: object
    properties:
      value:
        type: number
    required: [value]
    additionalProperties: false

  - type: object
    properties:
      value:
        type: string
    required: [value]
    additionalProperties: false
...

Now the following will fail:
with asdf.AsdfFile(
    tree={"obj_a": OneOfClass({"value": "a string"})},
    extensions=[CustomExtension()],
) as ff:
    ff.write_to("oneOf.asdf")

jsonschema.exceptions.ValidationError: {'value': 'a string'} is valid under each of OrderedDict([('type', 'object'), ('properties', OrderedDict([('value', OrderedDict([('type', 'string')]))])), ('required', ['value']), ('additionalProperties', False)]), OrderedDict([('type', 'object'), ('properties', OrderedDict([('value', OrderedDict([('type', 'number')]))])), ('required', ['value']), ('additionalProperties', False)])

If I only keep the string part of the schema as the single oneOf entry, it works. (same when trying to store a number, only works when the number part is the only oneOf list entry)
oneOf:
#  - type: object
#    properties:
#      value:
#        type: number
#    required: [value]
#    additionalProperties: false

  - type: object
    properties:
      value:
        type: string
    required: [value]
    additionalProperties: false
...

When changing oneOf to allOf I get jsonschema.exceptions.ValidationError: 'a string' is not of type 'number'
Using anyOf in the original example works.
While I can get by using anyOf I would prefer oneOf for clarity and peace of mind. Any hints are appreciated.

Thanks for the report!  This looks like an asdf bug.  I was able to successfully validate your example instance and schema using the jsonschema library directly, so asdf must be muddying the issue somehow.  I should be able to look into this later today.

The validation failure is raised by the asdf code for removing default values, and not the actual validation pass.  That code uses the oneOf combiner to traverse the schema, but disables all actual validation, so oneOf ends up failing because all subschemas report zero errors.
Unfortunately while we have a flag to disable filling defaults on read, we have no such flag to disable removing defaults on write.  I think until we get this fixed your best option is to get by with anyOf.
Incidentally, we're planning to deprecate the feature that fills/removes default values from schemas.  If a real fix turns out to be high effort, we may want to just add the missing flag instead.  Would that work for you?

Thank you for looking into this @eslavich
I almost suspected some interference with default values after looking through other issues but my understanding is too limited to dig into it myself.
I will stick with anyOf for now and try to keep each case as tight as possible in cases where I would otherwise put oneOf.
I have not really used or relied on default values so far but putting a flag should not be a problem.

@CagtayFabry the fix for this turned out to be straightforward.  It's in master if you'd like to give it a try.

I just found that PR 811 have made the following schema invalid:
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "example.com/schemas/example/nddataframe-1.0.0"
title: |
  An n-dimensional index frame.
description: |
  This frame is an abstraction of the indices of an n-dimensional array.

type: object
properties:
  name:
    description: |
      A user-friendly name for the frame.
    type: string

  anyOf:
    - naxes:
      description: |
        The number of axes.
      type: integer

    - axes_order:
        description: |
          The order of the axes.
        type: array
        items:
          type: integer

anyOf:
  - required: [name, naxes]
  - required: [name, axes_order]
additionalProperties: true
...

The error happens when parsing the anyOf in properties. The subschema is found to be the list of that anyOf around asdf/schema.py, line 191, function validate_remove_default.

Hi @Jerry-Ma this is not a valid schema.  anyOf is not permitted in a list of properties.  I think this may be equivalent:
type: object
properties:
  name:
    description: |
        A user-friendly name for the frame.
    type: string

  naxes:
    description: |
      The number of axes.
    type: integer

  axes_order:
    description: |
        The order of the axes.
    type: array
    items:
        type: integer

anyOf:
  - required: [name, naxes]
  - required: [name, axes_order]
additionalProperties: true

This schema will validate any of the following:
name: some name
naxes: 4

name: some name
axes_order: [1, 2, 3, 4]

name: some name
naxes: 4
axes_order: [1, 2, 3, 4]

but will fail this:
name: some name

Great. That makes a lot sense. Thanks!
The reason that I was asking is that the schema in my original post worked before the PR #811, so I did think it was the schema itself was not valid from the first place.


the authors key is an array ...

eslint is complaining about one-liners in the code.
Off-topic here but does this rule make sense?

@cboulanger : With function closed {} yes - it's more readable.

Doesn’t qx lint --fix help?

Before run qx lint --fix you must run:
qx pkg update
qx pkg install

Merged, thank you @oetiker !


Improve printing of Results Page

The "Printable format" item on the Tools menu should be replaced by a simple Print command/button. This should generate the printable version of the current page, not as a page in its own right but as the content to be sent to the standard print dialogue. In other words, it should behave like every other print command on the web.

I have worked out how to print the page directly, using a print CSS stylesheet to control what is output. This is a simpler approach conceptually than the current strategy of generating a whole 'print-friendly' page (and then leaving the user stuck on it). There is a simple 'Print' button:

which brings up the standard print dialogue directly:

The main challenge is in writing CSS which cleanly filters out the unwanted content, given the baroque nature of the HTML which Rails generates.

Could we consider creating an icon for Print option instead of a button? I think it's more visual than a button.

Two arguments against having a print icon are aesthetics (I think the row of buttons looks neat, and corresponds to what the other two projects have) and potential accessibility issues.

As regards the CSS issue, it occurs to me that if we invent classes 'no-print' and 'print-only', and apply them to the generated HTML, it makes it very simple to crop out the page components which we don't want in the printed output (and also to declare new content which is intended only for the printed version of the page). This whole approach is something of a departure from what we have done in the past, and I would welcome @Vino-S 's views on it. My own view is that it's a simplification, and that it is more in line with standard web publication practice.

@richardofsussex I thought we agreed on the the results page wireframe where download and print icons are presented. Since this suggestion came from Amir, I think everything is fine from UX UI point. Personally, I also support icons for these two options as they are more visual. A long row of buttons doesn't seem like the most modern way to present functionality and doesn't increase appeal or simplicity. If this issue is controversial, we can discuss it again together with Amir at the next meeting.

Interesting. @AlOneill can clearly see your image of the wireframe, but I get "Non-Image content-type returned" when I click on the link.

Just to say that I have managed to clear all the extraneous material from the print page:

@richardofsussex Thank you! Works on test3. Ready for deployment.


Avoid permanently switching to foreground window keyboard layout on Windows

In e1ee48fa20, we started switching to the foreground window keyboard
layout when using some of the APIs. This was done to ensure that we'd
get the main process keyboard layout (which is what Electron windows
would use) even when the APIs are called in the render process.
This works fine if the foreground app is the same as the caller of these
APIs, but if the foreground app is something else, we are inadvertently
switching to its keyboard layout. This can happen e.g. if some Electron app
calls these APIs while it's in the background.
This goes against Windows principles — the user may choose to use
different layouts in different apps.
Ideally we should remove this code and recommend that these APIs are
only used in the main process, but for now, we can avoid breaking
changes by simply switching back to the original layout after the
relevant keyboard querying code has executed.

Hm, created this PR just to trigger CI, but guess GitHub Actions workflows don't run on forks?


Image resice recoursively generate thumbnails until fails with too long filename

[REQUIRED] Step 2: Describe your configuration

Extension name: _ (storage-resize-images)
Configuration values (redact info where appropriate):
Deployment locationDescription
europe-west2
Cloud Storage bucket for imagesDescription
packedoo-dev.appspot.com
Sizes of resized imagesDescription
200x200
Cloud Storage path for resized images (Optional)Description
Parameter not set
Cache-Control header for resized images (Optional)Description
Parameter not set

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:
Installed image resize extension with default settings, Uploaded image to the default bucket,
thumbnails started to be generated recursively until failed because of too long file name
(https://firebase.google.com/docs/extensions/manage-installed-extensions#view-logs), etc.
Expected result
Actual result

Hey @kirillshilin - this sounds like an issue which was discovered and solved in the 0.1.1 release. Can you confirm which version you're using?

Hey @kirillshilin - if you're still having issues on an upgraded version of the extension, please let me know & this can be reopened and investigated. Thanks!


Hash collision triggers infinite recursion in eq()

If two distinct expressions have the same hash, computing the equality expression does not terminate:
from blaze import data, compute

x = 1
y = 2**63-3
assert hash(x) == hash(y)
xd = data(x, dshape='datetime')
yd = data(y, dshape='datetime')

compute(xd == yd)

The infinite loop is entered here:

241         def _resources(self):
242             return toolz.merge([arg._resources() for arg in self._args
243                                 if isinstance(arg, Node)])
244
245         def _subterms(self):
246             return subterms(self)
247
(Pdb) [arg._resources() for arg in self._args if isinstance(arg, Node)]
[{<'int' data; _name='None', dshape='datetime'>: 1}, {<'int' data; _name='None', dshape='datetime'>:<PHONE_NUMBER>854775805}]

toolz.merge() uses a new equality operation because the key hashes are equal.  This operation calls __eq__() with the exact same parameters as before, so there's an infinite loop.

@skrah see this comment and following: https://github.com/blaze/blaze/pull/1348#issuecomment-175833936
specifically:

For this to collide is extremely unlikely. I am not sure how we can ever stop hash collisions like this when we have overridden __eq__. This is not a new issue to blaze though, we use the hash of nodes for many api's including the scopes used in the core compute code so I am not sure that this has ever been an issue.

From what I recall on that thread, I believe the fundamental issue is due to the fact that Blaze overrides __eq__ for expressions, so whenever there is a hash collision, the fallback __eq__ comparison isn't testing for expression equality.
So as long as we have overridden __eq__ to create a new expression, we're stuck with this broken edge case.  From a pure design point of view, it's trading correctness for convenience.  My preference would be to revert this, but it's pervasive in Blaze, as @llllllllll points out on that thread, so we'd have to rip out the guts of Blaze to fix it, and the new interface wouldn't be as easy to use.

What if we make the hash the id of the node. Now that we have node identity caching this should not collide when the nodes are not the same

This won't help when the node is checked against a non-node, but it should help the blaze spaceI think

@llllllllll that seems to work, and I agree it would help with Blaze objects at least.  Performance-wise, the test suite runs in the same amount of time using id(self).
@skrah can you try @llllllllll 's suggestion and see if it resolves the test failures you're seeing?

@kwmsmith @llllllllll I have some problems with the suggestion, but perhaps I did not understand it fully. This is what I used:
diff --git a/blaze/expr/core.py b/blaze/expr/core.py
index 2e5c311..2648e16 100644
--- a/blaze/expr/core.py
+++ b/blaze/expr/core.py
@@ -201,7 +201,7 @@ class Node(object):
             hash_ = _setattr(
                 self,
                 '_hash',
-                hash((type(self), _static_identity(self))),
+                id(self),
             )
         return hash_

Then compute(xd == yd) in the original example code erroneously returns True instead of entering an infinite loop.

Actually I think the real problem is that blaze relies on name to be set correctly. This works:
from blaze import data, compute

x = 1
y = 2**63-3
assert hash(x) == hash(y)
xd = data(x, name='xd', dshape='datetime')
yd = data(y, name='yd', dshape='datetime')

compute(xd == yd)

If the names aren't set, https://github.com/blaze/blaze/blob/master/blaze/compute/core.py#L330 happily uses None as the symbol names, which leads to both problems in this issue.
Since to me the signature data(x, name='xd', dshape='datetime') implies that name is optional, perhaps the solution is to generate unique symbols if it isn't given, e.g. __anon123.
Additionally, of course swap_resources_into_scope could also check for None and raise an error.

@kwmsmith   Sorry, I mixed up two distinct issues in the previous message. Indeed, @llllllllll's suggestion fixes the hash related test failure from #1531.
I'll open a new issue for the behavior of swap_resources_into_scope.


MI/WI - Generate secrets for platform identities

[!CAUTION]
HOLD this PR until #3619 is merged and this PR is rebased.
The changes in #3619 are currently included in this PR's branch.

Which issue this PR addresses:
Fixes no Jira yet
What this PR does / why we need it:

Adds a SecretLocation coordinate to PlatformWorkloadIdentityRole, that holds a NamespacedName reference to the default Secret location for each platform workload identity. These coordinates are intended for use internally, and will not be exposed on the public ARO API.
Adds a function generatePlatformWorkloadIdentitySecrets to the cluster manager for eventual use during ARO cluster installation (to generate secrets to pass to the installer) and during ARO updates (to directly update secrets on-cluster). Note that the current implementation as-is is unused anywhere within the codebase, and will be used in eventual work items ARO-4518 (create) and ARO-4371 (update).

Test plan for issue:

[x] Unit tests were added for this direct functionality and they pass

Is there any documentation that needs to be updated for this PR?
No
How do you know this will function as expected in production?
This implementation as-is goes unused in any production contexts. It will eventually be utilized in MIWI cluster installation and update flows (see above).

/azp run ci

/azp run ci

/azp run e2e

/azp run ci,e2e

LGTM, just a comment on additional manifests needed.
There are 2 more manifests:-
`
apiVersion: config.openshift.io/v1
kind: Authentication
metadata:
name: cluster
spec:
serviceAccountIssuer:
apiVersion: v1
stringData:
azure_tenant_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
kind: Secret
metadata:
name: azure-credentials
namespace: openshift-cloud-credential-operator
type: Opaque
`
Can/Should we handle these two at the RP side and in the same PR?

LGTM, just a comment on additional manifests needed. There are 2 more manifests:-
apiVersion: config.openshift.io/v1
kind: Authentication
metadata:
  name: cluster
spec:
  serviceAccountIssuer: <OIDC ISSUER URL>
---
apiVersion: v1
stringData:
  azure_tenant_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
kind: Secret
metadata:
  name: azure-credentials
  namespace: openshift-cloud-credential-operator
type: Opaque

Can/Should we handle these two at the RP side and in the same PR?

For the Authentication resource, if it's a static, single resource as it appears to be, I think it can be done with a similar pattern to what's done here but it doesn't need to happen directly in this PR. Do we need to update this resource during cluster update as well (can the OIDC issuer URL change)?
For the CCO secret - I thought cloud-credential-operator would be included as a required platform workload identity and therefore have an entry in the roleset, meaning this PR would generate that secret with coordinates to a dedicated identity, is that not the case? Running oc adm release extract --from=${RELEASE_IMAGE} --credentials-requests --to=./scratch/credentials-requests --cloud azure gives me a CredentialsRequest resource for CCO so I assumed it'd get a dedicated identity.

Do we need to update this resource during cluster update as well (can the OIDC issuer URL change)?
No OIDC issuer URL update is not in scope currently, but the ensure function for the same can help here?

For the CCO secret - I thought cloud-credential-operator would be included as a required platform workload identity and therefore have an entry in the roleset, meaning this PR would generate that secret with coordinates to a dedicated identity, is that not the case? Running oc adm release extract --from=${RELEASE_IMAGE} --credentials-requests --cloud azure gives me a CredentialsRequest resource for CCO so I assumed it'd get a dedicated identity.
Tried executing oc adm release extract --cloud=azure --credentials-requests quay.io/openshift-release-dev/ocp-release:4.14.0-rc.6-x86_64 and it doesn't return the credential requests for CCO. The secret for CCO will not be part of the dedicated required Identities and secret will need to be created explicitly, as CCO doesn't require to access Azure resources, but it uses the TenantID for some of it's operations, probably related to setting up the webhook.

/azp run ci,e2e

/azp run ci,e2e

LGTM, since it is dependent on https://github.com/Azure/ARO-RP/pull/3619, whenever it is merged this can be rebased and unblocked.

/azp run ci,e2e

/azp run ci,e2e

/azp run ci,e2e

/azp run ci,e2e

/azp run ci,e2e


Overlapping transparent shader materials cancel out in GD 4 beta 10 but not 9

Godot version
v4.0.beta10.mono.official [d0398f62f]
System information
Ubuntu 22.04
Issue description
This may not be an issue, as I'm very new to shaders, but I had noticed that the behavior in two of my shaders has changed between betas 9 and 10.
I have a water shader, then a separate shader applied to a plane in front of the camera to add a water effect when the players camera goes underwater.
In beta 9, everything works as expected.  You can see the warping effect of the screen overlay as well as back, non-culled face of the water surface.  In beta 10, you can only see the screen overlay, and the water surface completely disappears.
Once again, this may be due to an issue in my shaders, so keep that in mind.  Also it's important to note that this behavior only applies to the back face of water mesh.  The behavior is the same (not blending the effects) for the front face in both betas.
Beta 9:

Beta 10:

Steps to reproduce
Simply open up the attached project and view the main.tscn scene from the camera.  Half the camera is covered with the screen overlay, and half isn't.  In beta 9, you can see both effects, in beta 10, you can only see one.
Minimal reproduction project
bug.zip

I'm not sure if this is a bug. I think this is a change in behaviour introduced by https://github.com/godotengine/godot/pull/69998, but I am fairly certain the behaviour in beta 9 was a bug. If you take a look at the scene in beta 9 you will see that the overlay you have added only effects the portion of the screen not covered by the "water" block.
So in Beta 9 what is happening is the overlay is drawn first, then the water is drawn second which draws over the overlay (i.e. the overlay does not affect the water block). Although from some angles the water is properly drawn first which causes the overlay to flicker in and out.
Beta 9

In Beta 10 and above, the overlay appears to be properly sorted so that it always draws after the "water" block. This causes it not to display the water as its entire color comes from the SCREEN_TEXTURE which only captures opaque objects.
To illustrate what is going on I modified the shader code for the overlay to tint it yellow. Here is how it looks in Beta 9:

And in Beta 10/11:

Note how in Beta 9 the water is drawn overtop of the overlay and in Beta 10/11 the overlay is properly drawn on top

I was suspecting something along those lines.  Thanks for looking into it.


As a user, I want to be able to add new vocabulary as it is introduced in the Cherokee Language Lessons 1 book, so that I can be working on the same terms in many contexts

https://github.com/CherokeeLanguage/online-exercises/pull/7

Confident that #7 covers this.


AttributeError: 'TransformerDecoderLayerOptimal' object has no attribute 'self_attn'

When i try to train the model on MSCOCO2014, i have this error that i can't fix : AttributeError: 'TransformerDecoderLayerOptimal' object has no attribute 'self_attn'... what should i do ?

I have meet the same question.

It has been resolved when I change Pytorch version from 2.1.1 to 1.13.1.

But when i do that i have the error : "ModuleNotFoundError : No module named 'torch._dynamo'...

Did you solve that ?

But when i do that i have the error : "ModuleNotFoundError : No module named 'torch._dynamo'...
I don't have that problem. I can run this project after change Pytorch version.

Found a fix. Use vanilla nn.TransformerDecoderLayer instead of the custom TransformerDecoderLayerOptimal  in Line 124 of ml_decoder.py:
https://github.com/Alibaba-MIIL/ML_Decoder/blob/8a9e984f671c9c30c98d2c45dfcaf4383381c254/src_files/ml_decoder/ml_decoder.py#L124
layer_decode = nn.TransformerDecoderLayer(
    d_model=decoder_embedding,
    nhead=8,
    dim_feedforward=dim_feedforward,
    dropout=decoder_dropout
)

Did you solve that ?

Install PyTorch 1.7 to solve the issue


Update framework and dual monitor support required

It should work on current systems without the need for additional framework installation and there is no dual monitor support. These updates will be very useful.

Yes, you're completely right, being a tool for Screenshots, it shouldn't be required something like a framework. I am aware of the dual monitor lacking but actually, this project is not maintained for now. I want to do this project again on another language/framework that allows LiveShot to be used without any external framework, maybe using Rust or C++.
Thank you very much for your feedback!

We met earlier on google talk. If you add these supports, we may pay you some fee.

Just use LightShot

Just use LightShot

LightShot gives me error when trying to download and I found this. Is there future in this project? Because I can't even launch it, it requires NET and wouldn't download it by itself. And I have .NET last version. Thats the problem.


LinkFix: windows-driver-docs-ddi (2021-11)

This PR is the result of running a Link Repair script on the included content. This script is primarily being run to clean up links to function correctly in Air Gapped Clouds (AGC). Here's a breakdown of what the script that generated the changes in this PR fixes:

docs.microsoft.com Fully Qualified Domain Name (FQDN) Links to either Relative (if in same Repo/DocSet/ContentSet) and Site Relative if not

azure.microsoft.com/documentation/articles that reflect to Docs to either Relative (if in same Repo/DocSet/ContentSet) and Site Relative if not

All known flavors of MSDN/TechNet domains (we know of 8 or 9 now) that reflect to Docs to either Relative (if in same Repo/DocSet/ContentSet) and Site Relative if not

This also includes cleanup of the appended query string for MSDN (redirectedfrom=MSDN).

Fixes articles that are redirected in .openpublishing.redirection.json and updates the link to the current final location (this has a lot of customer usability fixes).

Please note the Contributor Guide: Links has been updated with the following link-type preference order:

By order of preference, links hosted on Docs should be Relative if they are in the same repository and docset. If they are in a different docset, even if in the same repository, they should be Site Relative. Links to content hosted on Docs shouldn't use a complete URL, otherwise known as Fully Qualified Domain Names (FQDN). Using a complete URL from Docs to other content on Docs will cause that link to be non-functional in air-gap environments.

@mhopkins-msft - just FYI, this PR updates some URLs to use windows hardware instead of the relative dir reference.
HIDP_BUTTON_CAPS

Thanks @cmcclister!


only 2 file on google drive can be proces

hi, thanks for the great script but when I test using files on drive google, it only can process 2 file from many files on it,
I test it using the lite one

Thanks for bringing it up. I know there's something not right with batch process google drive files but to be honest couldn't get my head around this problem yet.

maybe you can look into script from Meizhong : https://github.com/meizhong986/WhisperJAV/tree/main


RuntimeError: You need to use the eventlet server.

RuntimeError: You need to use the eventlet server. See the Deployment section of the documentation for more information.
Error on request:
Traceback (most recent call last):
File "/home/eric/.local/lib/python3.8/site-packages/werkzeug/serving.py", line 319, in run_wsgi
execute(self.server.app)
File "/home/eric/.local/lib/python3.8/site-packages/werkzeug/serving.py", line 308, in execute
application_iter = app(environ, start_response)
File "/home/eric/.local/lib/python3.8/site-packages/flask/app.py", line 2088, in call
return self.wsgi_app(environ, start_response)
File "/home/eric/.local/lib/python3.8/site-packages/engineio/middleware.py", line 63, in call
return self.engineio_app.handle_request(environ, start_response)
File "/home/eric/.local/lib/python3.8/site-packages/socketio/server.py", line 589, in handle_request
return self.eio.handle_request(environ, start_response)
File "/home/eric/.local/lib/python3.8/site-packages/engineio/server.py", line 392, in handle_request
packets = socket.handle_get_request(
File "/home/eric/.local/lib/python3.8/site-packages/engineio/socket.py", line 103, in handle_get_request
return getattr(self, 'upgrade' + transport)(environ,
File "/home/eric/.local/lib/python3.8/site-packages/engineio/socket.py", line 158, in _upgrade_websocket
return ws(environ, start_response)
File "/home/eric/.local/lib/python3.8/site-packages/engineio/async_drivers/eventlet.py", line 16, in call
raise RuntimeError('You need to use the eventlet server. '

try this? it said uninstall eventlet

It worked! Haha, thanks for the help! :)


parseExcludes in fractal is supported?

Summary of problem or feature request

I understand that laravel-datatables supports Fractal Transformer, as suggested in this issue , but is there a way to use Excluding Includes feature in laravel-datatables, and how?
Thanks!

It's not yet supported atm. Will try to implement this or if you can, PR's are welcome. Thanks!

I made a quick look on parseExcludes and I think it's not implemented on fractal package?
Call to undefined method League\Fractal\Manager::parseExcludes()

@yizhihou2 , I think fractal excludes is not yet implemented on that package and I got the error above when trying to implement it. Anyways, you can now extend the fractal manager by accessing it through the IoC app('datatables.fractal') and extend it as needed. PR #636 released on v6.15.1. Thanks!


Keep Awake - option for 'until another program is not busy'

Description of the new feature / enhancement
I would like to be able to select another option on the Keep Awake tray icon context menu for 'keep awake until program finishes'
Then, I choose the process from a list or maybe just select an active window on the screen like a terminal window.
Once the selected program exits or its cpu utilization drops near zero for a consistent amount of time then the pc can go to sleep as normal.
Scenario when this would be used?
My most common reason for using Keep Awake is I need my pc to stay awake while some process like a large file copy operation is in progress or a media transcoding process. Currently I will chose 'indefinitely' however then my computer will just be left on, sometimes for day or so. Or I go for the max of 2 hours and the pc sleeps before the long running operation finishes.
Supporting information
No response

Leaving this open for now to get more input, but I am hesitant to commit to anything that attempts to understand the "busyness" of an application. That's mainly because it's such an ambiguous concept - what's the threshold for "busyness"? Is it I/O activity beyond a certain volume? % of CPU or CPU core usage? RAM used? And there is a myriad of reasons why an application might see spikes or drops in activity unrelated to the core action it's performing.
@kimmccoy - is binding to a process, and then waiting until that process exits not sufficient?

I don't think waiting for a process to exit would be enough. Keep awake is handy for applications and processes that aren't nice to work with. So for example my backup software will provide options to shutdown when done, but other programs like some transcoding ones just take ages and finish with an open window.
So, same issue as https://github.com/microsoft/PowerToys/issues/35406  if you want to close.
I concede it is probably impossible to judge when a program is idle 'enough' in a reliable way that has a nice simple UI.


Flywheel has failed to load correctly

Describe the Bug
As soon as I open the CTE2 modpack it gives me the error as soon as it loads open MC.
Reproduction Steps

Press "Play"
Wait for MC to do initial loading
Shows error

...
Expected Result
The modpack to work
Screenshots and Videos

Crash Report or Log
https://pastebin.com/sNe7QFe5
Operating System
Windows 10
What is your GPU?

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

This is a forge issue caused by the ~ in the file path, you've got a folder named ~Game either rename that to Game or place this minecraft in another folder that doesn't have a ~ in it's path

Thanks that fixed it


OpenGL on ARM Chromebooks  ?

Hey,
Sorry to be vague in my question title, basicly I need some clarification.
I understand that when using Xephyr (Which is what we use on ARM chromebooks); there can be no acceleration.
Say I get the lib's (libgl etc ...) bundled with the driver.
My questions are as follows :

If it were possible to run a second X Session, would that session be 3d accelerated ?
If that is not possible, would it be possible to use a console (tty) to kill the main X server, then chroot and launch a replacement ?
Could that have 3d acceleration ?

I really appreciate the efforts of the developers who put this together, currently my XFCE chroot is brilliant, but deep down I yearn for Gnome Shell ;)
Thanks to anyone who can respond :)

Which ARM chromebook do you use? It's shown that the Acer Tegra K1 don't support second X Session with hardware acceleration. However xorg can be installed rather than Xephyr. It shows a better graphic performance. I'd like to close main X server and start one for chroot.

I've got the HP Chromebook 11 (daisy spring)
I was just curious as to if it was possible. so far i haven't worked out how to keep Xorg from restarting automatically on chromeos .. working on it :)

That would be #964.  Separate X11 is not an option yet.


#12 bugfix: limit fps to 30

I have used the pygame.time.Clock().tick() function to limit the frame rate of the game to 30. This way the game would be the same speed regardless of the system it is being run on.

I personally found the game too easy at 30 FPS! What do you think about customizing FPS as per difficulty (see #9)?