Deadlock between onStreamCompleted and Session.Close

There appears to be a deadlock between calling Session.Close(...) and various Stream calls (eg Read) which may trigger a call to onStreamCompleted while holding a Stream lock.  They each hold a lock the other needs since they acquire them in different orders (eg legacy_stream_map lock and receive_stream lock)
Here is a (finnecky, contrived) reproduction of a particular instance of this problem: deadlock_test.go
When it hits the issue, the goroutines are deadlocked like this:
Full output
goroutine 107673 [semacquire, 9 minutes]:
sync.runtime_SemacquireMutex(0xc420740384, 0x0)
    /usr/local/Cellar/go/1.10/libexec/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc420740380)
    /usr/local/Cellar/go/1.10/libexec/src/sync/mutex.go:134 +0x108
sync.(*RWMutex).Lock(0xc420740380)
    /usr/local/Cellar/go/1.10/libexec/src/sync/rwmutex.go:93 +0x2d
github.com/lucas-clemente/quic-go.(*streamsMapLegacy).DeleteStream(0xc420740380, 0x2d, 0x0, 0x0)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/streams_map_legacy.go:224 +0x39
github.com/lucas-clemente/quic-go.(*session).onStreamCompleted(0xc420711400, 0x2d)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/session.go:1219 +0x3e
github.com/lucas-clemente/quic-go.(*stream).checkIfCompleted(0xc420654000)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/stream.go:173 +0x6f
github.com/lucas-clemente/quic-go.newStream.func2()
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/stream.go:119 +0x52
github.com/lucas-clemente/quic-go.(*uniStreamSender).onStreamCompleted(0xc420590000, 0x2d)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/stream.go:41 +0x29
github.com/lucas-clemente/quic-go.(*receiveStream).Read(0xc420654000, 0xc420870000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/receive_stream.go:161 +0x606
io/ioutil.devNull.ReadFrom(0x0, 0x2a8c4c0, 0xc420654000, 0x12aa680, 0xc42094cf01, 0x2a0a170)
    /usr/local/Cellar/go/1.10/libexec/src/io/ioutil/ioutil.go:147 +0x92
io.copyBuffer(0x1335fe0, 0x14c5860, 0x2a8c4c0, 0xc420654000, 0x0, 0x0, 0x0, 0x12592e5, 0xc420478800, 0x0)
    /usr/local/Cellar/go/1.10/libexec/src/io/io.go:386 +0x31a
io.Copy(0x1335fe0, 0x14c5860, 0x2a8c4c0, 0xc420654000, 0x0, 0x0, 0x37)
    /usr/local/Cellar/go/1.10/libexec/src/io/io.go:362 +0x5a
created by github.com/ltucker/quic_deadlock.TestSessionDeadlock.func1
    /Users/ltucker/go/src/github.com/ltucker/quic_deadlock/deadlock_test.go:75 +0x257

goroutine 106267 [semacquire, 9 minutes]:
sync.runtime_SemacquireMutex(0xc420654004, 0xc4204b9a00)
    /usr/local/Cellar/go/1.10/libexec/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc420654000)
    /usr/local/Cellar/go/1.10/libexec/src/sync/mutex.go:134 +0x108
github.com/lucas-clemente/quic-go.(*receiveStream).closeForShutdown(0xc420654000, 0x13357e0, 0xc4208d2460)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/receive_stream.go:267 +0x2d
github.com/lucas-clemente/quic-go.(*stream).closeForShutdown(0xc420654000, 0x13357e0, 0xc4208d2460)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/stream.go:153 +0x6a
github.com/lucas-clemente/quic-go.(*streamsMapLegacy).CloseWithError(0xc420740380, 0x13357e0, 0xc4208d2460)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/streams_map_legacy.go:256 +0x13d
github.com/lucas-clemente/quic-go.(*session).handleCloseError(0xc420711400, 0x1335f80, 0x0, 0xc4204b9c00, 0x27f00dca4, 0x14a83c0)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/session.go:839 +0x16f
github.com/lucas-clemente/quic-go.(*session).run(0xc420711400, 0x0, 0x0)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/session.go:535 +0xeec
github.com/lucas-clemente/quic-go.(*client).establishSecureConnection.func1(0xc420804820, 0xc4205cce00, 0xc4200b65a0)
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/client.go:251 +0x41
created by github.com/lucas-clemente/quic-go.(*client).establishSecureConnection
    /Users/ltucker/go/src/github.com/lucas-clemente/quic-go/client.go:250 +0xa3

Hi @ltucker, thanks for reporting this! The test case is super helpful.
I'm currently traveling, so it might take a couple of days until I can review your PR.


Page does not load if CalendarView's IsVisible property is set.

Hi there,
I seem to be having a problem with the control where if I set the IsVisible property on it , the ContentPage hosting it will not load. I have tried this with a new AppShell tabbed project and a new App Project. With the former the tabour comes up, but the tab of the page that hosts the control does not load. With the latter, the page just doesn't come up at all.
Both tests use the simple example shown on the getting started page here repeated below:
VM
public Calendar<CalendarDay> MyCalendar { get; set; } = new Calendar<CalendarDay>();
Xaml
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage
    x:Class="App1.MainPage"
    xmlns="http://xamarin.com/schemas/2014/forms"
    xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
    xmlns:xc="clr-namespace:XCalendar.Forms.Views;assembly=XCalendar.Forms">

        <xc:CalendarView
            Days="{Binding MyCalendar.Days}"
            DaysOfWeek="{Binding MyCalendar.DayNamesOrder}"
            NavigatedDate="{Binding MyCalendar.NavigatedDate}"/>

</ContentPage>

As far as I can see, this issue has not been reported yet, but apologies for any trouble if I have missed it. I have added the AppShell project here and the CalendarView is used on the  ItemsPage.Xaml and its corresponding ItemsViewMode view model.
Any help will be much appreciated.
Kind regards
Test details:
Platform: Xamarin Forms (<IP_ADDRESS>62)
Calendar (4.6.0)
xCalendarTest.zip

Issue in XCalendar for future reference:
ME-MarvinE/XCalendar#187


chore(package): Babel and Rollup configuration.

Closes https://github.com/styled-components/cynosure/issues/9

@mxstbr copied the rollup configuration over from styled-components. Removed any dependencies on React for this build since it should not be needed. I also copied the babel.rc that was created, looks like it was addressing an issue in styled-components, let me know if you think that won't be needed for this.

@mxstbr gonna have to look at this a bit more in the morning. Fixed some issues with capitalization of file names in the repo causing things to work locally but not on Travis. But now running into an issue where Jest passes on both src and lib locally, but fails on src on Travis. Looks like it is not using es6 properly on Travis for some reason.

@mxstbr nevermind, figured it out. Added babel-jest to the test suite, so now we can write our tests in ES6 as well. Odd that it would work locally without it, but could be to me having babel installed globally. This is ready to merge.

Awesome, thanks!


Travis CI: test against node.js 4.

I also let Travis ran with 0.10 and 0.12 just to test if things still work there; everything went fine so we can keep the engines value we set in package.json to 0.10. Although it could break at any time and we wouldn't notice...
/CC @cvrebert

I also let Travis ran with 0.10 and 0.12 just to test if things still work there; everything went fine so we can keep the engines value we set in package.json to 0.10. Although it could break at any time and we wouldn't notice...

Perhaps we should bump the minimum engine to 0.12? Or maybe even 4?

@cvrebert: Personally, I'd go with the version(s) we test on CI for the reason I mention in my OP. But, I see how that might be too strict.

Personally, I'd go with the version(s) we test on CI for the reason I mention in my OP. But, I see how that might be too strict.

I agree on both counts. Bootstrap v4 already breaks backward compatibility and Node.js v4 is installable on all major platforms, so it seems like we might as well upgrade, but I think we need more folks to weigh in before making the call.
CC: @twbs/team for more input

Bump @twbs/team for feedback.

I'd argue for supporting 0.12 until it reaches its EOL. It is currently in LTS:

My main concern is that we only test 1 version with Travis which means things might break without us knowing. But I don't think limiting to good either; it's too strict.

Perhaps we should inquire what libsass's Node.js version compatibility policy is, since they likely put a floor on what Node.js version(s) we can support.

@cvrebert @hnrch02 : should I change this to stable now that Travis CI supports that? Or keep using 4?

Ping. Now that we have node.js 5.0 out we need to at least use the LTS 4.x release IMO.

Opened https://github.com/sass/node-sass/issues/1247 to ask what node-sass' policy is.

My main concern is that we only test 1 version with Travis

Well, we could use Travis' matrix feature to run the core build on multiple Node.js versions. Doesn't look too hard to setup.

I'd argue for supporting 0.12 until it reaches its EOL. It is currently in LTS:

0.12's LTS period is a strict subset of v4's LTS period, so supporting it wouldn't buy us any extra LTS time...

Well, you didn't want to test multiple versions. And to be honest it's not of a great use for us; I feel like using the latest stable is enough. But being this is a CI, we should test 4 and stable at least.

So, based on the discussion on the node-sass issue, node-sass doesn't impose much of a restriction on us; they support Node.js 0.8 thru latest Stable Node.js, which is even wider than @hnrch02's suggestion. (At least so long as there's no major Node.js/V8 shakeup that'd force NAN to narrow its compatibility.)

So, should we test 0.10, 4 and stable and see how times are on Travis?

Well, you didn't want to test multiple versions.

If I did say that at some point, I hereby rescind it.

I feel like using the latest stable is enough

I imagine the pathological case here is someone running a (fleet of) website with a Node.js-based backend (*barf*) that wants to compile Bootstrap in the same server process (e.g. "Adjust these Bootstrap variables on this Settings page to customize the look of your website!"). (It's suboptimal and kinda weird, but not patently crazy.)
And since it's a server, they're slower to update Node.js. Such folks would therefore appreciate Active LTS support.
@XhmikosR So I agree with your recommendation of 4 and stable.
@hnrch02 Your thoughts?

@cvrebert 4 and stable sound good.

Switched to 4 and stable.


A problem occurred evaluating project ':react-native-vision-camera'🐛

What were you trying to do?
I was running my app without issue with the version 2.10.0, but since the update of 2.11.0 the build failed with an exception.
on the /react-native-vision-camera/android/build.gradle' line: 8
it's looking for the file /react-native-vision-camera/node_modules/react-native/ReactAndroid/gradle.properties
which doesn't exist (No such file or directory).
I tried to clean the caches but still happening

Reproduceable Code
"react-native-vision-camera": "^2.11.0",

What happened instead?
FAILURE: Build failed with an exception.
Relevant log output
FAILURE: Build failed with an exception.

* Where:
Build file '/Users/yohanbalassiano/Sites/Deelt/app/deelt/node_modules/react-native-vision-camera/android/build.gradle' line: 8

* What went wrong:
A problem occurred evaluating project ':react-native-vision-camera'.
> /Users/yohanbalassiano/Sites/Deelt/app/deelt/node_modules/react-native-vision-camera/node_modules/react-native/ReactAndroid/gradle.properties (No such file or directory)

Device
OnePlus Nord
VisionCamera Version
2.11.0
Additional information

[ ] I am using Expo
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

Hi, thank you for the report.
That bug has already been reported in a comment before, it's because the path to the node_modules folder seems to be wrong. I will take a look soon, for now you can hotpatch it using patch-package :)

Okay had some free time to investigate this - can you please try if #704 works for you?

Hi @mrousavy, I had the same issue. When using the fix/monorepo-support I get the following.
FAILURE: Build failed with an exception.

* Where:
Build file '/home/jens/projects/craftzing/extraHorizon/fibricheck_react_native/node_modules/react-native-vision-camera/android/build.gradle' line: 24

* What went wrong:
A problem occurred evaluating project ':react-native-vision-camera'.
> /home/jens/projects/craftzing/extraHorizon/fibricheck_react_native/node_modules/react-native-vision-camera/node_modules/react-native/ReactAndroid/gradle.properties (No such file or directory)

If I change line 19
-def nodeModules = findNodeModules(projectDir)
+def nodeModules = findNodeModules(rootDir)

It does build

Maybe your node_modules/react-native-vision-camera folder also has another node_modules/ folder inside? if yes, delete that.
I'll try your suggested solution if that works here ...

Let's see if #706 builds in the CI.

Thanks  @jensdev  its work for me


Lidar image problem

Could anyone tell me why there is such a big difference between my LiDAR images and the author's?

Hello , Is your CARLA version 0.9.6? Looking at this difference may be that your version is not compatible.

Hello , Is your CARLA version 0.9.6? Looking at this difference may be that your version is not compatible.

Yes, this is a problem due to the Carla version. I've tested it on 0.9.6, it works well.
However, we would like to use it on the latest version since other part of our work is already done. I've modified some incompatible api to make the environment error-free, but this lidar display is still wrong (with 0.9.13, it's basically black, you can only see some edges of the pavement). While I'm trying to figure out what changed (data order? data meaning?）, I would like to know if somebody has already been through this.

Hi,  I am sorry I have not met this problem as you said that ''with 0.9.13,it's basically black...", but I've successfully test the project on 0.9.13, from 0.9.6 to 0.9.13 version, I only add "client.get_trafficmanager().set_synchronous_mode(True)" in  ''carla_env.py'' with comments ''Set fixed simulation step for sychrounous mode'' this line. Good luck.
------------------ Original ------------------
From:                                                                                "cjy1992/gym-carla"                                                                                @.>;
Date: Wed, Nov 2, 2022 10:16 PM
@.>;
<EMAIL_ADDRESS>Subject: Re: [cjy1992/gym-carla] Lidar image problem (#31)
Hello , Is your CARLA version 0.9.6? Looking at this difference may be that your version is not compatible.
Yes, this is a problem due to the Carla version. I've tested it on 0.9.6, it works well.
However, we would like to use it on the latest version since other part of our work is already done. I've modified some incompatible api to make the environment error-free, but this lidar display is still wrong (with 0.9.13, it's basically black, you can only see some edges of the pavement). While I'm trying to figure out what changed (data order? data meaning?）, I would like to know if somebody has already been through this.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

Hi, I am sorry I have not met this problem as you said that ''with 0.9.13,it's basically black...", but I've successfully test the project on 0.9.13, from 0.9.6 to 0.9.13 version, I only add "client.get_trafficmanager().set_synchronous_mode(True)" in ''carla_env.py'' with comments ''Set fixed simulation step for synchronous mode'' this line. Good luck.

Unfortunately, that doesn't help. The only part I've changed relating to rendering lidar data is changing to location.point.x/y/z at this line:
https://github.com/cjy1992/gym-carla/blob/c01ae77d2729c618504595474b744a1641ec471f/gym_carla/envs/carla_env.py#L495-L496
By "basically black", I mean I can't see the red rings like this:

And I've noticed that it seems like the heading of lidar data is not right, it's facing right instead of up.

Hi, I am sorry I have not met this problem as you said that ''with 0.9.13,it's basically black...", but I've successfully test the project on 0.9.13, from 0.9.6 to 0.9.13 version, I only add "client.get_trafficmanager().set_synchronous_mode(True)" in ''carla_env.py'' with comments ''Set fixed simulation step for synchronous mode'' this line. Good luck.

Unfortunately, that doesn't help. The only part I've changed relating to rendering lidar data is changing to location.point.x/y/z at this line:
https://github.com/cjy1992/gym-carla/blob/c01ae77d2729c618504595474b744a1641ec471f/gym_carla/envs/carla_env.py#L495-L496
By "basically black", I mean I can't see the red rings like this:

And I've noticed that it seems like the heading of lidar data is not right, it's facing right instead of up.

I have the same problem here. Looking forward to solutions for this issue.

@YanlinQi @Morphlng Hi, they may try to change this line
for location in self.lidar_data:    point_cloud.append([loaction.x, location.y, -location.z])
to the line point_cloud.append([loaction.x, location.y, location.z]) or point_cloud.append([loaction.point.x, location.point.y, location.point.z]),
I've successfully test my project in this setting.

@ZuojinTang Thanks a lot for providing this solution. It indeed works! I have been wondering why there is a "-" before location.z, but didn't try it out. Many thanks to you!

try to change this line :  point_cloud.append([loaction.x, location.y, -location.z]) to the line: point_cloud.append([loaction.x, location.y, location.z]) or point_cloud.append([loaction.point.x, location.point.y, location.point.z]),

By changing this line, the lidar image no longer seem black, yet the direction is still incorrect. The BEV and WayPoints is vertically displayed but the lidar image is horizontally displayed, a screenshot below can show this more directly.

I think there is still something wrong about those flip and rot90, I'm still trying to fix it.

try to change this line :  point_cloud.append([loaction.x, location.y, -location.z]) to the line: point_cloud.append([loaction.x, location.y, location.z]) or point_cloud.append([loaction.point.x, location.point.y, location.point.z]),

By changing this line, the lidar image no longer seem black, yet the direction is still incorrect. The BEV and WayPoints is vertically displayed but the lidar image is horizontally displayed, a screenshot below can show this more directly.

I think there is still something wrong about those flip and rot90, I'm still trying to fix it.

I just tried to change the rotation of y, and it works for the rotation problem:

self.lidar_trans = carla.Transform(carla.Location(x=0.0, y=0.0, z=self.lidar_height), carla.Rotation(0, 90, 0))

Brothers, I have solved this problem, you need to adjust the following parameters，my carla is 0.9.13
'd_behind': 16,  # distance behind the ego vehicle (meter)
x_bins = np.arange(-(self.obs_range - self.d_behind), self.d_behind+self.lidar_bin, self.lidar_bin)
y_bins = np.arange(-self.obs_range/2, self.obs_range/2+0.01, self.lidar_bin)

Brothers, I have solved this problem, you need to adjust the following parameters，my carla is 0.9.13 'd_behind': 16, # distance behind the ego vehicle (meter) x_bins = np.arange(-(self.obs_range - self.d_behind), self.d_behind+self.lidar_bin, self.lidar_bin) y_bins = np.arange(-self.obs_range/2, self.obs_range/2+0.01, self.lidar_bin)

Awesome! It works pretty well now. I am highly appreciated for that!

朋友你也复现他这个论文吗，要不要认识交流一下啊，我这里遇到一些问题

兄弟，我已经解决了这个问题，你需要调整以下参数，我的carla是0.9.13 'd_behind': 16, # ego vehicle后面的距离（米） x_bins = np.arange(-(self.obs_range - self .d_behind), self.d_behind+self.lidar_bin, self.lidar_bin) y_bins = np.arange(-self.obs_range/2, self.obs_range/2+0.01, self.lidar_bin)

惊人的！ 现在效果很好。 对此我深表感谢！

兄弟，我已经解决了这个问题，你需要调整以下参数，我的carla是0.9.13 'd_behind': 16, # ego vehicle后面的距离（米） x_bins = np.arange(-(self.obs_range - self .d_behind), self.d_behind+self.lidar_bin, self.lidar_bin) y_bins = np.arange(-self.obs_range/2, self.obs_range/2+0.01, self.lidar_bin)

惊人的！ 现在效果很好。 对此我深表感谢！

朋友你也复现他这个论文吗，要不要认识交流一下啊，我这里遇到一些问题

兄弟，我已经解决了这个问题，你需要调整以下参数，我的carla是0.9.13 'd_behind': 16, # ego vehicle后面的距离（米） x_bins = np.arange(-(self.obs_range - self .d_behind), self.d_behind+self.lidar_bin, self.lidar_bin) y_bins = np.arange(-self.obs_range/2, self.obs_range/2+0.01, self.lidar_bin)

惊人的！ 现在效果很好。 对此我深表感谢！

朋友你也复现他这个论文吗，要不要认识交流一下啊，我这里遇到一些问题

兄弟，我已经解决了这个问题，你需要调整以下参数，我的carla是0.9.13 'd_behind': 16, # ego vehicle后面的距离（米） x_bins = np.arange(-(self.obs_range - self .d_behind), self.d_behind+self.lidar_bin, self.lidar_bin) y_bins = np.arange(-self.obs_range/2, self.obs_range/2+0.01, self.lidar_bin)

惊人的！ 现在效果很好。 对此我深表感谢！

朋友你也复现他这个论文吗，要不要认识交流一下啊，我这里遇到一些问题

好啊，怎么加你呢^_^


Should PostgresCompiler quote identifiers by default?

Throwing this out as a discussion topic because as far as I can find it hasn't been mentioned before.
A quirk of PostgreSQL is that quoting identifiers affects the behavior for how identifiers are handled.  Quoting from the Postgres docs:

Quoting an identifier also makes it case-sensitive, whereas unquoted names are always folded to lower case. For example, the identifiers FOO, foo, and "foo" are considered the same by PostgreSQL, but "Foo" and "FOO" are different from these three and each other.

AFAIK, it's usually considered a Postgres best practice to avoid to avoid quoting identifiers unless you must use a name that collides with a reserved keyword.  I don't know the history of SqlKata, but I suspect that like many .Net tools it probably started out focused on the MS stack (SQL Server) and was eventually expanded to support other systems.  And of course in T-SQL delimiting an identifier with square brackets doesn't change any behavior, and it's not unusual to preemptively put delimiters around everything "just in case."
So given that context, the default behavior of PostgresCompiler can be a little annoying.  For example if you have a DTO whose property names match your column names (using a non case-sensitive comparison) and you want to dynamically generate select statements using the property names, it simply won't work - you have take the extra step of transforming the names to lower case yourself.  This feels like an unexpected behavior since I think most people are used to SQL being case-insensitive, and it can be confusing since many people aren't intimately familiar with the rules for quoting names in Postgres (they've just been taught not to do it).  However it's not a show-stopper, since most people who are just typing column names in their code probably won't run into a problem.
Honestly I feel like the quoting behavior should probably be removed from PostgresCompiler entirely.  I'm struggling to think of a scenario where I would want identifiers to be quoted by default.  However this would be a breaking change if anyone is relying on that behavior, so instead perhaps introduce a property to control it so that something like new PostgresCompiler { QuoteIdentifiers = false } is available.  I'd also suggest a callout in the documentation recommending to disable quoting unless you specifically need the legacy behavior.

I'm in a situation where we require identifiers to be unquoted, for both SQL Server and Postgres. Is that possible with SQLKata?
Situation: Migrating from SQL Server to Postgres, lowercasing identifiers during the migration, but we have a reporting system with hardcoded, mixed-case identifiers. Excluding quotes will allow both databases to work with a lowercased schema. We do not have any identifiers which require quoting.

Replying with an answer to my own question: you can implement a custom compiler pretty easily:
https://stackoverflow.com/questions/66053712/sqlkata-is-there-any-way-to-tell-the-sqlkata-compiler-to-not-wrap-identifiers
Unfortunately, this requires the 3.0.0 beta


Implement setSlider

Tested on:
model='Mito sospeso move up', manufacturer='Occhio'

I'm not sure that slider will always mean height. Could you give me the unit id for this light? I'd like to check if there is something in the unit specification that indicates that slider means height in that case.

Thanks for following up!
This is the information of the device in the network JSON:
{
    "deviceID": 1,
    "uuid": "xxxxxxxxxxxxxx",
    "address": "xxxxxxxx",
    "cpu": 1,
    "name": "Mito sospeso move up",
    "type": 16006,
    "appearance": 0,
    "hueID": 0,
    "groupID": 0,
    "position": 0,
    "revision": 1,
    "configRevision": 1,
    "firmware": 11008,
    "modes": [
        {
            "name": "unit_mode1",
            "state": "00ff7f7f00",
            "cycle": true
        },
        {
            "name": "unit_mode2",
            "state": "00007f7f00",
            "cycle": false
        },
        {
            "name": "unit_mode3",
            "state": "00007f7f00",
            "cycle": false
        },
        {
            "name": "unit_mode4",
            "state": "00007f7f00",
            "cycle": false
        }
    ],
    "smartSwitch": {
        "type": 0,
        "id": 0,
        "scenes": [],
        "scenes2": [],
        "group": 0,
        "unit": 0,
        "lingerTime": 120,
        "fadeTime": 10,
        "switchOffFadeTime": 10,
        "switchOffDelayTime": 0,
        "excludeOFF": false,
        "disableDimming": false,
        "presetAction": 32,
        "presetTarget": 0,
        "index": 0
    },
    "pushButtonToggleDisabled": false,
    "linkedSensor": 0,
    "gain": 100.0,
    "startupMode": 0,
    "minOnLevel": 0.0,
    "minLevel": 0.0,
    "maxLevel": 1.0,
    "parameters": [],
    "behaviour": 0,
    "timeout": 0,
    "labels": {},
    "settings": {
        "supportsLongRange": 0.0,
        "cct.minKelvins": 2700.0,
        "cct.maxKelvins": 4000.0
    },
    "details": {},
    "actionConfig": 0,
    "actions": []
}

The settings in the application:

The control is defined as follows:
{"type":"slider","name":"$control_pos_h","unit":"cm","offset":32,"length":8,"min":50,"max":200,"default":0,"readonly":false,"valueType":1,"flags":0}
This means its a custom control and sliders might have different meaning based on model. I'm not sure whether logic required to fully support custom sliders is really necessary (and this would probably necessitate some API changes). However, I think some more logic is needed to detect that a slider is actually a height control. I'm also not quite sure how to determine the correct opcode for height adjustment, but I guess the current one is fine until someone complains about issues with a unit with multiple sliders.
@michieldwitte Do you want to have a go at implementing this (and can you follow my thoughts)? Otherwise I'll take over this PR and will try to get this done for the next release. The infrastructure for supporting additional numerical controls on the HA side is already finished and planned for the next release.

Ok that solution is also fine (and easier). Thanks for your contribution.


Added toggle button

Description
This PR implements a light/dark mode toggle feature as described in issue #95. The toggle allows users to switch between Light Mode and Dark Mode across the editor application, and the theme choice is saved in localStorage to persist across sessions.
Changes Made

Added light and dark mode CSS styles.
Implemented a toggle button to switch between modes.

Pls let me any other change required here.
Thanks

Hi @kk7188048 please check your code commits eslintrc etc. and make sure the above errors are resolved

@kk7188048 You need to run npm run lint:fix locally.


Change Model Orientation for Thumbnail

All my models are facing away in thumbnails. Can an option be added to change the orientation/facing direction?

That is how the creator has created them. There is no way that I can find
how the orientation looks best. The "thumbnail" shown in Windows File
Explorer is a static picture. If you choose to have my STL viewer as
prefered application for STL, simply doubble clik on the STL and turn the
model
Regards
Den sön 26 aug. 2018 kl 17:42 skrev danexpo<EMAIL_ADDRESS>
All my models are facing away in thumbnails. Can an option be added to
change the orientation/facing direction?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cabbagecreek/STL-thumbnail/issues/3, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAriYS9srQOuUQAo792UutyUN4hvtVFfks5uUsHdgaJpZM4WM1DW
.

I think that @danexpo is right and there is a problem with orientation - if i open the same STL file in any other editor / viewer / slicer, they are "right", here they are rotated in Z-axis by 180°. That makes it often almost unusable (e.g. when i got a set of minifigures and i need to quickly find the right one)

Ok.. Give me more info about this.. Like pictures and actual STL files that I can test with.
What other stl viewer do you use?
What happens in cura, Simplify3D etc
I can se if this is a generic problem and if I render the model with the wrong camera angel/light

Thanks for the great app, but I have the same issue.  In Windows "Paint 3D", Windows "Print 3D", Windows "3D Viewer", Cura, and Simplify3D the STLs looks fine.  But in your app, they are backwards.

Thats actually how I Parse the STL. There is no standard in whats up/down/Right/left in STL.. So I just chose the ”Top” as being the level plane.
This will be the wrong position for 1-of-3 cases 😉
There is a big problem in STLÖ. Everybody like STL standard , so then the programmer of different software invent their own STL standard.
The code is actualli in GIT and if you have C# skills you are free to help me find how to correct this.
Give me mor info and the STL that you have found working in ”Paint3D” etc and I will look at this. The biggest problem is not to render tha model. Its to get Windows to accept the ”File Explorer Shell Extension”. This is the main reason why noone has tried before. Its almost impossible to get Windows to accept the new ”filetype” stl and render it in File Explorer.
Rhe reason why I did this is because I don´t want to be associated with any software. This is inside Windows and independent on any type of software. I took me about 9 month of programming to get this close. ( Close… but no cigarr )
Regards
Johnny
Skickades från E-post för Windows 10
Från: Rick Tremmel
Skickat: Sunday, 20 October 2019 00:37
Till: cabbagecreek/STL-thumbnail
Kopia: Johnny Lindén; Comment
Ämne: Re: [cabbagecreek/STL-thumbnail] Change Model Orientation forThumbnail (#3)
Thanks for the great app, but I have the same issue. In Windows "Paint 3D", Windows "Print 3D", Windows "3D Viewer", Cura, and Simplify3D the STLs looks fine. But in your app, they are backwards.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

It seems like this behavior stems from treating Y as up, when most models use Z as up.
The similar Papa's Best STL Thumbnails project defaults to Z up, which is the behavior preferred by the participants here. It is honestly also quite a bit faster and displays colored vertexes and materials as well.


[Addition]:

Hotspot b58 Addresses
11eStNppx8Q4rUEcxzVsNK8sP87f5cb9PVCtoJXNRRpMcTTwqLK
114gND4TRiCqd6WhuGo6VrD2cK4Js9LtcqFi2VCaN5cBfUWgJSH
11B5NxfexRtXxykpX7bnpR7Mh7TQbScTz8QoBJqz1ecJd8wcTLL
112PgbRcxtvDxuwPyakEt299zTQZynLdainVcdd4gjSrz6CuoXW7
1183hLQTKDwjM6NQq96KPbn1gECg2mT1fPiqVqTdsC6Nfx9ae2k
112Zfqzrd4GfdLkPywHU3Ff91qww4nLdUcExmaqbJuUHNUzTsCzj
112oDFWFynEdHrXgmS46TkXVqsuWA3jDuN1eryfNYQZymTUw2FQk
11JHdaEXtiEkzTc5khQ5KPV1CxKci91FWBgFEQ16bgVcPj8PjjT
11mNcJ9oSYzjwhTMcCKUQNPM9GxUPnAJfUQx4LeegKCsWBzStA5
11pRGsTeC8KnFitsWEG4t7mnxywwzuJiKMNmjg2ReMN54DuphaQ
115JjZEku5WQnXi7EadtPHvjWk9XGBLx6i5VyKEUvJwYZMRhm1u
112SRvY5uyFyj3Wcom2WqNcuQ7PYza9uC5qysTSTPNJGoZBDngzz
112YzADqAgGx9t6n2USGG2KLLhbGg8QNE4casCXopBrG1PysUvpD
112aLw9U61FgCF4B7yZpBmssjGrrssfR3RB7hgmQeEPM8aZG19vE
Reason(s)
And again Pisces farm. This is only small part of the addresses from that area, but  all of these Pisces miners are not there.
PLEASE DO SOMETHING WITH THESE MOTHER FUCKERS!!!!!!!!!!!

We have taken into account this addition request in the next iteration of the denylist. Some or all of the Hotspots listed may be added to the denylist. This is an automated comment due to the volume of addition and removal requests. A future tool is in the works to provide more insight into the analysis approach made by the Helium team and community members that maintain this list.


Deploy Redash application

Deploy the full stack of Redash services using Pulumi and launching the Redash AMI

Don't see any obvious issues so 👍


WIP: feat: initial setup logo and colors

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

this is awesome! can't wait to start filling this up with additional components.
just left a comment about license update. could you also sign the CLI?


How can I add custom blocks?

Where must I put the code to add custom blocks. I did't find any solution for this problem.

Hello! At the moment there is no convenient API for OctoberCMS. But you can extend it by manually editing gutenberg.js in /formwidgets/gutenberg/assets/js. For more information how to add a custom block, read here https://github.com/VanOns/laraberg#custom-blocks


Add Backup-DbaServiceMasterKey command

Type of Change

[ ] Bug fix (non-breaking change, fixes #)
[x] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[x] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose

Add Backup-DbaServiceMasterKey command for backing up the service master key (as proposed in #5397
Approach

Copied, then modified the Backup-DbaDbMasterKey.
Commands to test

(These are the examples)
Backup-DbaServiceMasterKey -SqlInstance server1\sql2016
Backup-DbaDbMasterKey -SqlInstance Server1 -Path \\nas\sqlbackups\keys

Screenshots

Learning

yessssssssss! I was hoping to see this one! thank you so very much :bowtie:

This is my first time trying to contribute a new command, so I'm a little iffy on the tests/ if the command name needs added to dbatools.psm1.
Another big ? is the auto-generated filename for the export. I went with
<instance>-SMK-<timestamp>.key.


Genericise list indexing for T: Num

From the WIBNI pile. It'd be nice if I didn't have to have so many different implementations of list indexing.

It looks like this may not be possible without some rather gross casts. I'll see what I can do.

Comparing disparate types T : Num would require an external dependency and probably make indexing too slow. Closed as wontfix.


Edit: HTV4 to HTV Key

Channel ID (required)
HTV4.vn
Channel Name
HTV Key
Alternative Names (optional)
No response
Network
No response
Owners
No response
Country
No response
Subdivision
No response
City
No response
Broadcast Area
No response
Languages
No response
Categories
No response
NSFW
None
Launched
No response
Closed
No response
Replaced By
No response
Website
No response
Logo
https://upload.wikimedia.org/wikipedia/commons/9/9e/Logo_HTV_Key.png
Notes
HTV4 became HTV Key in 2018, idk why it's still HTV4 in 2022

Fixed in https://github.com/iptv-org/database/commit/562073fd2b33494118c8941be7d067880494be07


[bug] Flexbox disappears

Describe the bug
After having AI generate some code and applying flexbox, the controls disappear. When I try to unapply it or select a different layer, the UI bugs out.

https://github.com/user-attachments/assets/8d37bee2-fe8a-487b-b340-a1649f55eb49

This should now be solved. Please reopen if not.


WIP: centosci-tasks-h943k: Check for NPM dependency updates

Check for any outstanding update of point version of any outstanding NPM package dependency

[ ] npm-update

npm-update in progress on centosci-tasks-h943k.
Log: http://fedorapeople.org/groups/cockpit/logs/npm-update-566-20190304-070808/


Fix for #5238. Count was broken when counting new vertices in same tr…

Please also merge into develop!

I didn't see this. Merged the old one, should be the same.

Yep, new one was just a cherrypick of the old one.


BXMSPROD-1340 kogito-tooling nightly

https://issues.redhat.com/browse/BXMSPROD-1340
I didn't find a better way to avoid --registry=X flag error running yarn build:prod
[INFO] Running 'yarn build:prod --registry=RH_repo' in /home/jenkins/workspace/PROD_kogito-tooling.nightly_main/kiegroup_kogito-editors-java/kogito-editors-js
[INFO] yarn run v1.22.10
[INFO] $ rimraf dist && yarn format:check && lerna run build:prod --stream --registry=https://repository.engineering.redhat.com/nexus/repository/registry.npmjs.org/
[INFO] $ prettier --check .
[INFO] Checking formatting...
[INFO] All matched files use Prettier code style!
[INFO] ERR! lerna Unknown argument: registry
[INFO] error Command failed with exit code 1.
[INFO] info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[webpack-cli] Error: Unknown option '--registry=RH_repo'
[webpack-cli] Run 'webpack --help' to see available commands and options
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

it is true additional arguments won't be taken into consideration in case build:prod from packages change... any better idea?

@Ginxo For me this work on qe jenkins:
export NPM_REGISTRY=REG
npm config set registry=${NPM_REGISTRY}
yarn config set registry ${NPM_REGISTRY}
locktt --registry=${NPM_REGISTRY}

yarn bootstrap && yarn build:prod

@Ginxo For me this work on qe jenkins:
export NPM_REGISTRY=REG
npm config set registry=${NPM_REGISTRY}
yarn config set registry ${NPM_REGISTRY}
locktt --registry=${NPM_REGISTRY}

yarn bootstrap && yarn build:prod

thanks @domhanak. The problem is about running maven https://github.com/kiegroup/kogito-editors-java/blob/b2b1786dc2019c539b0c75403cc9cbc7d828d4a0/kogito-editors-js/pom.xml#L23 with npm registry on maven settings.xml so --registry is set on every npm command from frontend-maven-plugin

If you rerun FDB it should be green, the error is fixed on main.


Fix typo and update doc on Canary App metrics

Signed-off-by: Glenn Oppegard<EMAIL_ADDRESS>

Thanks, @oppegard!


Single unified config aggregator

Use case

When developing a cross-platform app using Flutter, updating the app's version number, permissions, icons, splash screens, and package/bundle names for both iOS and Android can be time-consuming and labor-intensive, as they need to be managed separately for each platform. This can cause inefficiencies and increase the chance of errors in the development process. It also makes it difficult to keep track of the differences between the two platforms, maybe even more than two!
Proposal

I propose the development of a unified config file to simplify the management of version information, permissions, icons, splash screens, and package/bundle names across multiple platforms such as iOS, Android, MacOS, Windows, etc. This file would enable developers to efficiently manage all of these settings in one central location, eliminating the need to update them separately for each platform. Additionally, it would make it easy to keep track of variations between platforms, reducing the potential for errors.
The config file could be in a format such as JSON or YAML and include fields for each of the settings that need to be managed, such as app version, package name, and icon file path. This file would be placed in the project's root directory and the build process would automatically read from it to update the relevant files for each platform.
For example, let's say that you are developing a cross-platform app using Flutter that requires access to a user's photos. Currently, in order to add this permission, you would need to update the AndroidManifest.xml file for Android and the Info.plist file for iOS separately. This can be a time-consuming and error-prone process, especially if you are working with multiple developers on the same project.
With a unified config file, you would simply add a field in the config file for "photo access permission" and set it to "true" or "false" depending on whether or not the app needs to access the user's photos. Additionally, you can also have a field for the permission description that will be displayed to the user when the app requests access to the user's photos, this would make it more transparent and user-friendly.
This feature would significantly save time and reduce the chance of errors during development, making it easier for developers to maintain cross-platform apps using Flutter. It would also make it much simpler for new developers to update the config for each platform without the need to navigate through each platform individually.
It is worth noting that some of the information listed, such as the version and icons, are already present in pubspec.yaml or with flutter packages from pub.dev. However, my proposal suggests a unified interface for all config that can be generated by the flutter cli, which would scan each platform folder and output all configs. This can be further enhanced with a UI extension in VSCode, to make it more convenient for developers to update the config file for all platforms or to assign different values for specific configurations on different platforms. This config file can be generated from an existing project or updated and applied to a new project.
This can also allow developers to share their config in tutorials or in a package on pub.dev.
This might be a lot but since iOS, Andriod, etc platforms configs do not change that frequently the benefits might outway the cost of developing such a thing.

Keeping this issue open and labeling for team's thoughts on this proposal.

using pkl can be a way to achieve some of the goals of that issue
(eg: version number, package/bundle names and permissions)
but all (it wont help for icons, splash screen, etc)


[Feedback] Alfsteel feeling lackluster

I've been messing with the mod for a while and just wanted to offer some feedback on the tools. Overall, they're feeling somewhat lackluster.
Take the armor, for instance. It is, for all intents and purposes, the same as Terrasteel. Yes, you get a minor runspeed, jump, and reach bonus but someone who's using Botania heavily already has access to these sorts of bonuses with other trinkets.
Compare this to the other armors:
The Manaweave set which offers a massive bonus to tool usage, but very little protection
Manasteel offers greater protection but less of a bonus to mana.
Elementium is greater protection still and has the very unique bonus of summoning the fairy sprites.
Each has it's own unique flair to it. Something that makes it stand out from the other armors. Alfsteel just seems like it needs something a little more unique too. Also, I'm not sure I follow the massive increase to durability as these things self repair with mana anyway.
Then there's the tools.
Alf blade? Bigger numbers
Alf Shatterer? Bigger area (diminishing the value of the Gaia relics)
Alf Truncator? Built in magnet when a magnet ring is cheap to make?
They all just feel like they're missing something special. Something unique. Something that would make the player actually want to spend the mana to make them.
I think the Alfsteel Pylon achieves this quite nicely, filling a niche that nothing else in Botania really does: Repairing non-botania items. This feels like a wonderful addition and I can see myself using it in the future.
I'm honestly not certain what to suggest as suitable ideas here. I guess the question comes down to this: What's special about Alfheim? What sort of theme should it impart on it's materials? What kind of story is being told?
Anyway, thanks for reading :D I hope you take this in the positive spirit it was meant.

Suggestion for alf blade. The projectile should pierce and/or ricochet between mobs.

@yuriy145 piercing through mobs will be added in the 1.18 update.


Using Feign to get Response Headers

Hi!
I'm using Spring Cloud (v Brixton.SR7) with Hystrix + Feign for my requests. (v 1.2.3.RELEASE).
According to this, I should be able to use ResponseEntity as the method return, to retrieve the headers in a response.
However, if I use that that, I get the following error:
feign.FeignException: Can not deserialize instance of org.springframework.http.ResponseEntity out of START_ARRAY token  at [Source: java.io.BufferedReader@45f2337a; line: 1, column: 1] reading GET <<<URL>>>
I'm using ResponseEntity<MyObject> as I get this error. If I replace it with MyObject, the deserialisation is successful. So apparently it's not a problem with the model, but something with Feign not able to build a ResponseEntity?
I couldn't find any of this in the Spring Cloud documentation nor the Feign documentation. I assume this is a common use case (reading headers), so maybe it'd be nice to include something to that extend there.
Am I doing something wrong? Any info I failed to provide? What's the proper way of getting the response headers with Spring Cloud + Hystrix + Feign?
Thank you in advance

Could you perhaps provide your Feign client implementation for reference?

The guys over OpenFeign gitter helped me out and I kind of solved it.
The correct object to return is Response (feign.Response). With that, you can access the headers. The only downside is that now the request body is an input stream and won't be deserialised for you anymore.
Ideally we would be able to use Response and Feign would do that job for for us.
But for now using it solves my problem of getting the headers.

He's not using the Spring response entity decoder


Issue with Placename being two words

usaddress.RepeatedLabelError:
ERROR: Unable to tag this string because more than one area of the string has the same label
ORIGINAL STRING:  101 W Avenida Vista Hermosa Suite 122 San Clemente CA 92672
PARSED TOKENS:    [('101', 'AddressNumber'), ('W', 'StreetNamePreDirectional'), ('Avenida', 'StreetName'), ('Vista', 'StreetNamePostType'), ('Hermosa', 'PlaceName'), ('Suite', 'OccupancyType'), ('122', 'OccupancyIdentifier'), ('San', 'PlaceName'), ('Clemente', 'PlaceName'), ('CA', 'StateName'), ('92672', 'ZipCode')]
UNCERTAIN LABEL:  PlaceName

I've run into the same issue with Ann Arbor. Except it is parsing "Ann" into the StreetNameSuffix column, leaving "Arbor" as the PlaceName.


Julia 1.6 / Win64 fix: use 1ms timeout hack on Win64

This shouldn't be necessary and doesn't seem to be on older Julia
versions, but for some reason on Julia master the first download
hangs forever on Win64 unless we do this the old hacky way.

Codecov Report

Merging #21 into master will decrease coverage by 2.33%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #21      +/-   ##
==========================================
- Coverage   96.86%   94.53%   -2.34%
==========================================
  Files           5        5
  Lines         255      256       +1
==========================================
- Hits          247      242       -5
- Misses          8       14       +6

Impacted Files
Coverage Δ

src/Curl/Multi.jl
98.76% <100.00%> (+0.01%)
:arrow_up:

src/Curl/utils.jl
82.05% <0.00%> (-15.39%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a551ad8...e0cfcbc. Read the comment docs.

Turns out this was not a bug here but a but in libcurl, fixed by upgrading it.


Can't import AWS EKS cluster in an VPC with public and private subnets.

Rancher Server Setup

Rancher version: 2.6.2
Installation option (Docker install/Helm Chart):

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): EKS

Proxy/Cert Details:

Information about the Cluster

Kubernetes version: 1.21
Cluster Type (Local/Downstream):

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider): Imported AWS EKS

Describe the bug
The cluster keeps in pending status.

Here is my EKS cluster

The account used to import EKS cluster is already have the "system:masters" permission of kubernetes, and I can access the cluster's api server on my local computer.

$ kubectl --kubeconfig=${HOME}/.kube/config-basicai-dev-rancher get node
NAME                                        STATUS     ROLES    AGE    VERSION
ip-10-0-29-218.us-west-2.compute.internal   NotReady   <none>   4h9m   v1.21.5-eks-bc4871b
ip-10-0-48-26.us-west-2.compute.internal    NotReady   <none>   4h9m   v1.21.5-eks-bc4871b
ip-10-0-85-219.us-west-2.compute.internal   NotReady   <none>   4h8m   v1.21.5-eks-bc4871b

I'm also tried the import generic way, but it's the same result. I  installed rancher by this guide Rancher on the AWS Cloud
, and I can access the rancher ui in my local browser, so there should be no problem with rancher itself. And I deployed the EKS cluster following this guide Amazon EKS on the AWS Cloud
.

Here is the api server's log in AWS CloudWatch.
E1208 07:41:56.654823       9 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, unknown]"
E1208 07:41:56.712853       9 webhook.go:127] Failed to make webhook authenticator request: unknown

I found the reason, because the node is not ready, after recreate the eks cluster, the node status become ready. But when I import eks cluster by the generic way, the cattle cluster agent log error "Get https://rancher.basic.ai: x509: certificate is valid for ingress.local, not rancher.basic.ai". I'm using an AWS NLB to allow access rancher from internet.

Finally I solved this problem! Because it's releated to the TLS certificate, so I applied an official business certificate (Digicert for free), and configure rancher ingress to use this certificate.


[libevent-2.1.10] arcticdb v4.4.0

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

That's the first time I see an message like https://github.com/conda-forge/arcticdb-feedstock/pull/192#issuecomment-2036623715.
It might be explained by your absence in the recipe maintainers on the libevent-2.1.10 branch which is corrected by a7ac933.


dotnet add fails

Following this page its not working out very well for me - see console output below:
PS Z:\dev\C#\BlobQuickstartV12> dotnet add package Azure.Storage.Blobs
Determining projects to restore...
Writing C:\Users\Mike\AppData\Local\Temp\tmpF515.tmp
error: There are no versions available for the package 'Azure.Storage.Blobs'.
Hope the feedback is helpful.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 60e56e69-1c44-2056-ad5f-319535e4b525
Version Independent ID: 49fa6098-0b8e-ebf5-efe7-f50f3604e325
Content: Quickstart: Azure Blob Storage library v12 - .NET
Content Source: articles/storage/blobs/storage-quickstart-blobs-dotnet.md
Service: storage
Sub-service: blobs
GitHub Login: @twooley
Microsoft Alias: twooley

@SinisterChinaPenguin Thanks for the question! We are investigating and will update you shortly.

@SinisterChinaPenguin It's mentioned in the article,

If you are facing any issue, please let us know this command automatically adds latest version of a Nuget package:

Yeah I get that bit - turns out the problem is I didn't have a nuget source to pull the package from. So I ran:
dotnet nuget add source --name nuget.org https://api.nuget.org/v3/index.json
& then it worked just fine. I guess maybe it's obvious to you guys but wasn't to me :-(
Thank You for getting back to me!

@ SinisterChinaPenguin We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Complete some missing specs

This is a follow up to #7052.
What was the end-user problem that led to this PR?
The problem was that the Bundle.with_clean_env & Bundle.with_original_env family was missing some deprecation specs, and also the specs were spread out across different files.
What is your fix for the problem, implemented in this PR?
My fix is to add the missing specs, and centralize them in the deprecation specs file.

Straightforward and affects tests only, so merging!
@bundlerbot r+


[Feature Request] Alternate Variation W/O System.Memory Reference and compat w/AOT/UNITY

As per the title, I am wondering if we may get NetStandard2.0 builds of MarkDig which do not reference System.Memory, or a build conditional which could be implemented to allow building from source to allow for builds which exclude optimizations which may rely on System.Memory and System.Buffers?
This is primarily for compatibility within Unity and it's supported platform targets.
To get around this, I currently use CommonMark, and intend to use it to to allow markdown formatting of In-game ui elements/quests, etc and to enable support for Markdown Files/Strings within unity's inspectors by translating HTML output into USS/UXML layouts.
System.Memory is an obstacle if we wish to build for WebGL and Mobile Platforms within Unity using NetStandard2.0, but is possible (with additional setup) to use when targeting .NET4.X on standalone (Desktop) platforms.
There are some other limitations which may affect Markdig compatibility with unity's NetStandard2 implementation (below).
My hope is for a variant of Markdig which will function out of the box in when taking the following into account

Targeting .NetStandard2/NET461 within Unity (manual create and edit csc.rsp, not guaranteed for all platforms)

Net4.X: Minimum target is 4.6.1,  maximum is a limited subset up to 4.7.

NetStandard: Only 2.0 is supported currently.

IL2CPP / AOT Platforms (with code stripping) - when using NetStandard2 while targeting WebGL

IL2CPP / AOT Platforms when targeting IOS/Android from within Unity (threading is possible).

Android: System.IO functionality is limited (UnityWebRequest  required to download from persistant data to a temp file before System.IO can be used to load a file)

(WebAssembly) WebGL, everything in System.Threading and System.Threading.Tasks is unusable

(WebAssembly) WebGL, all file operations and streams are unusable, UnityWebRequest for IO or Session/Cookies.

(NetStandard2) System.Buffers       unusable within Unity on all platforms

(NetStandard2) System.Memory  - depends on System.Buffers.

(NetStandard2) System.Text.Json    unusable within Unity  on all platforms

Other .Net assemblies require csc.rsp file edits by hand.

As per the title, I am wondering if we may get NetStandard2.0 builds of MarkDig which do not reference System.Memory, or a build conditional which could be implemented to allow building from source to allow for builds which exclude optimizations which may rely on System.Memory and System.Buffers?

We don't want to rollback the work that was done to improve performance, nor we want to maintain special case for older/constrained frameworks.
But you should be able to old this old version 0.18.3 of Markdig which didn't have these dependencies in netstandard 2.0.


Pattern Matching inconsistency

Compiler version
3.6.1
Minimized code
This does not issue a pattern match exhaustivity warning:
sealed trait T_A[B]
case class CC_C(a: Int, b: T_A[Int]) extends T_A[Char]

val v_a: T_A[Char] = null
val v_b: Int = v_a match{
  case CC_C(0, _) => 0
}

But this does:
sealed trait T_A
sealed trait T_B
case class CC_A(a: Int, b: T_B) extends T_A

val v_a: T_A = null
val v_b: Int = v_a match{
  case CC_A(0,_) => 0
}

Expectation
They should behave the same. The issue generally appears when matching against traits that don't have any case classes extending them, given certain generics. (Here T[Int]). But it does not appear when a trait simply has no case classes extending them, regardless of generics (T_B).
My guess is that this is related to the fact that the pattern matcher does not enforce covering nulls explicitly, so if the only way to create a case class is with a null it thinks that this case class could never be constructed (first program) and therefore also does not need to match against it
I believe my previous bug reports had this same issue at the root, and the fix in #21000 (which to my knowledge should be included in 3.6.1?) did not resolve this issue.

Could be related to the discussion at #21934


Use without Vue CLI

Hi!
I'm a big fan of quasar, it's have his own cli and would be cool to allow use this plugins without the Vue CLI requisite, so anyone can use it.
Thanks in advance 😄

Hi,
I'm planning on rewriting this plugin in TS, and to that end I extracted the main functionality into a separate Node module. It's yet undocumented but the API is the largely the same (you can look at the test cases to see some practical examples).

But it's ready now?

But it's ready now?

It's basically finished, it just needs to be documented.


fix(cli): make cicd operational while scaffolding via cli

Description
make cicd operational while scaffolding via cli
Fixes #1974
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Intermediate change (work in progress)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist:

[ ] Performed a self-review of my own code
[ ] npm test passes on your machine
[ ] New tests added or existing tests modified to cover all changes
[ ] Code conforms with the style guide
[ ] API Documentation in code was updated
[ ] Any dependent changes have been merged and published in downstream modules

can also add the yaml file that does the ci action on PR

can also add the yaml file that does the ci action on PR

hmm,
we can do that, will discuss on this in scrum


Gen 1: Full paralysis chance is 63/256

https://github.com/iimarckus/pokered/blob/master/engine/battle/core.asm#L3504

Why are there two pull requests for this?

@Irraquated: Separate mods, would say it's fine.

Ah, fair enough.

Yeah, I thought it would be better to commit one for each mod, but I was not sure about it. If changing both files in a single commit would be better next time, please tell me =)

@Guernouille it would make reviewing the pull request easier, in this case I guess it's fine because it's just changing a number.

Yes, both in a single commit would be preferred.


Issue 37067: Update grid related Data View thumbnail images

button bar text/icons have changed since last screenshots

Note: screenshots for updated images were taken from labkey.org demo study.


ptxas executable

I believe that the ptxas executable should be available here.
It seems that tensorflow (at least v1) attempts to use it.
However, when I create a fresh environment with cudatoolkit 11.2 it doesn't seem to be included.

[x] I read the conda-forge documentation and could not find the solution for my problem there.

Issue:

Environment (conda list):

$ conda list
# packages in environment at /home/mark/miniforge3/envs/cudatoolkit:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
cudatoolkit               11.2.0               h73cb219_9    conda-forge
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgomp                   11.2.0              h1d223b6_11    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_11    conda-forge

Details about  conda and system ( conda info ):

$ conda info

     active environment : cudatoolkit
    active env location : /home/mark/miniforge3/envs/cudatoolkit
            shell level : 3
       user config file : /home/mark/.condarc
 populated config files : /home/mark/miniforge3/.condarc
                          /home/mark/.condarc
          conda version : 4.10.3
    conda-build version : 3.21.4
         python version : 3.8.12.final.0
       virtual packages : __cuda=11.2=0
                          __linux=5.11.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/mark/miniforge3  (writable)
      conda av data dir : /home/mark/miniforge3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/mark/miniforge3/pkgs
                          /home/mark/.conda/pkgs
       envs directories : /home/mark/miniforge3/envs
                          /home/mark/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.26.0 CPython/3.8.12 Linux/5.11.0-40-generic ubuntu/20.04.3 glibc/2.31
                UID:GID : 1002:1002
             netrc file : None
           offline mode : False

xref: https://github.com/conda-forge/tensorflow-feedstock/issues/170

Currently cudatoolkit does not contain any executable or header from CUDA Toolkit because of the EULA limitation. The new package format (#62) will address this issue but I am unaware of any timeline for that.

Understood. Thank you for the explination

@leofang, I am a little confused about what is and is not available through the main cudatoolkit (the one in this stock). In particular:

is nvcc compiler available?
are things like cublas available?

I am primarily interested in essentially recreating what NVIDIA offers in their NGC containers in our packaging of tensorflow and pytorch; one of the key  missing items I've been working on is activating XLA for tensorflow, which requires the correct compilers etc. being available. Do you have any advice?

If not available, where are these things available in conda-forge? For example, I saw that some things are available through cudatoolkit-dev and I believe we have nvcc feedstock... Are we supposed to be using them that way? My understanding has been that these things should be bundled in cudatoolkit (as the first sentence above asserts)

cc @jakirkham for viz and comment

the offline compiler (nvcc) and headers are not available

on conda-forge, they are only available in the conda-forge's docker images that are deployed to the CIs for building packages
nvcc-feedstock is just a thin wrapper on top of the image's nvcc, no real use case outside of the CI AFAIK
to me cudatoolkit-dev technically violates the CUDA EULA, I don't know how it still exists, but perhaps it's not my business to ask 🙂

runtime shared libraries for cuBLAS etc are available

runtime shared libraries for cuBLAS etc are available

Where? Here?

the offline compiler (nvcc) and headers are not available

Okay, let me try to see what exactly is needed for tensorflow and pytorch and we can work on addressing these issues as they come.

on conda-forge, they are only available in the conda-forge's docker images that are deployed to the CIs for building packages

Since you talk about the EULA, etc. --- is using that conda-forge CI image for someone's production work okay or is it only for CI? I believe I saw it was based on the cuda-devel Docker images, so the licensing might be exactly the same as that (those cuda-devel images are the main building blocks for all NGC containers as far as I understand...)

Thanks for the prompt and clear answer by the way, 👍 @leofang

Btw, as far as I could tell, we are good for the XLA implementation, though I need to do more local testing to see if there are additional issues to resolve. @leofang, if you're interested in having a look, see https://github.com/conda-forge/tensorflow-feedstock

Where? Here?

Yes

Since you talk about the EULA, etc. --- is using that conda-forge CI image for someone's production work okay or is it only for CI?

I think it's OK. The CUDA images and their derivatives (including conda-forge's) are permissive. By using them users acknowledge the terms and conditions.

I have just encountered the issue that a particular TF model built from TF hub and run in Jupterlab in an Anaconda environment in which I had installed cuDNN and cudatoolkit raised issues because is missing from the conda-forge source.
I confess I cannot follow the discussion above. Can someone explain why this happened in CUDA toolkit 11.2 and what the future will be like?
It doesn't make sense to someone me: if the DLL's etc are available then why not the ptxas.exe??? I'm a user... I just want it to run. (NB finding out why ptxas was an issue and what to do about it was a PITA - now I have an installation, can I just copy ptxas into the environment somewhere appropriate??)
I had to install cuda in the OS, which undermines the value of having conda environments with the CUDA stuff in them.
Setup: win-10 home 64 bit 21H2.

@JulianSMoore What exactly is your error and what tensorflow are you using? Upgrade to 2.8.1 if you can!

Hmmm... view on github didn't seem to work, but here goes....
Error arose while executing this in a downloaded ViT (Vision Transformer
ipynb attached, without error info, showing working result after ptxas made
available)
model_url = 'https://tfhub.dev/sayakpaul/vit_s16_classification/1'

classification_model = tf.keras.Sequential(
[hub.KerasLayer(model_url)]
)
predictions = classification_model.predict(image)
predicted_label = imagenet_int_to_str[int(np.argmax(predictions))]
predicted_label

I can't recall (and didn't save :( ) the error details except that the
right ptxas was not found. I did find these notes though...
Then TF complains about the ptxas version (there are 2 on c: one at

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\ptxas.exe and
the other in
C:\Users\Julian\AppData\Roaming\Mathematica\Paclets\Repository\CUDAResources-Win64-10.5.0\CUDAToolkit\bin\ptxas.exe)

It says maybe I can just put the right version (?>11.1) in the cuda path,

i.e. into the OS CUDA 10

Obviously I wanted it to take it from the env, which conda install -c
nvidia cuda-nvcc achieved.TF is looking in the env first AFAICT, and then
falls back to an OS install - complex errors depending on env/os cuda
installation existence + version,
Setup is TF2.9.1
Name                    Version                   Build  Channel
tensorboard               2.9.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorflow                2.9.1                    pypi_0    pypi
tensorflow-addons         0.17.0                   pypi_0    pypi
tensorflow-datasets       4.5.2                    pypi_0    pypi
tensorflow-estimator      2.9.0                    pypi_0    pypi
tensorflow-gan            2.1.0                    pypi_0    pypi
tensorflow-hub            0.12.0                   pypi_0    pypi
tensorflow-io-gcs-filesystem 0.26.0                   pypi_0    pypi
tensorflow-metadata       1.8.0                    pypi_0    pypi
tensorflow-probability    0.16.0                   pypi_0    pypi
tensorflow-text           2.9.0                    pypi_0    pypi
cuda-nvcc                 11.7.64                       0    nvidia
cudatoolkit               11.2.2              h933977f_10    conda-forge
cupy-cuda112              10.5.0                   pypi_0    pypi
Python 3.10.4, Win-10 Home, 64 bit.
 Upgrade to 2.8.1 if you can!
—
Reply to this email directly, view it on GitHub
https://github.com/conda-forge/cudatoolkit-feedstock/issues/72#issuecomment-1144321215,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACHLK6N76DMHKBIRSEQLVP3VNAE2RANCNFSM5IRELETQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Not obvious to me that the notebook attached to email is available here, so attaching it separately in zip.
classification_ViT-checkpoint - Copy.zip

@JulianSMoore sorry we can't be of more help. A few things:

The anaconda channel is different than the conda-forge channel
The conda-forge channel still doesn't have many packages for cuda+windows
The conda-forge channels doesn't have tensorflow for windows
conda-forge is different from pypi and as such, the knowledge you gain here may not translate.

it seems that you are installing things through the Tensorflow recommended ways (pypi) and as such I would recommend you as a question on their forums. We simply don't have the knowledge to help you troubleshoot your system on windows.

@hmaarrfk Perfectly understood (wasn't expecting you to troubleshoot!) & your info will be helpful.
For the benefit of others: some required s/w support for TensorFlow (e.g. ptxas, for ViT model from TF hub) seems to lie outside cudatoolkit. If you encounter a similar issue, first check your paths, then think about the libraries used/needed and finally consider packages from different channels. Hard to be more specific than that, unfortunately. (I use conda installation for cuda tookit and cuDNN because that is the only way I know to install cuda in an anaconda env (rather than OS) - everything else I do with pip)

Isn't the main problem that the Nvidia EULA prevents us from distributing PTXAS and other binaries? We can't just ignore that. This all will go away when #62 lands and Nvidia officially distributes their packages on conda, which I assume would have a permissive enough license for us to redistribute. Til then... we can't do much, sorry.

Let's close this issue now that it is resolved with CUDA 12. Thanks everyone for the discussion and request.


spark connect fails when performing a .show()

Describe the bug
# %%
import daft
from daft.daft import connect_start
from pyspark.sql import SparkSession

server = connect_start()
url = f"sc://localhost:{server.port()}"
session = SparkSession.builder.appName("DaftConfigTest").remote(url).getOrCreate()

session.createDataFrame([("cory", 100)], ["name", "age"]).show()

results in "Error in Daft server: Unsupported relation type: ShowString"
To Reproduce
No response
Expected behavior
No response
Component(s)
Other
Additional context
It appears that currently our show logic exists purely in python. As a prerequisite, we'll need to refactor that logic into rust so that it can be used from spark connect.

Describe the bug
# %%
import daft
from daft.daft import connect_start
from pyspark.sql import SparkSession

server = connect_start()
url = f"sc://localhost:{server.port()}"
session = SparkSession.builder.appName("DaftConfigTest").remote(url).getOrCreate()

session.createDataFrame([("cory", 100)], ["name", "age"]).show()

results in "Error in Daft server: Unsupported relation type: ShowString"
To Reproduce
No response
Expected behavior
No response
Component(s)
Other
Additional context
It appears that currently our show logic exists purely in python. As a prerequisite, we'll need to refactor that logic into rust so that it can be used from spark connect.

also note createDataFrame with a string is currently bugged (I am working on fixing it)

Can we just use Display rust impl for now?

no, the logical plan display is equivalent to df.explain().
df.show() shows a small sample of the materialized dataset similar to df.limit(10).collect()

@andrewgazelka I can take on this one.

closed via https://github.com/Eventual-Inc/Daft/pull/3560


nixos/doc: fix build

Adjusts to options renamed in 90b1197301eb714e64a508c3be2dd073b24caed3

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Merging as is since it won't be able to check its affects.


[BUG] Incorrect JSON Structure in PATCH API

When using the API call function of a PATCH request, the generated JSON structure includes an extra "patchDocument" object wrapping the operations. It should instead directly contain the "operations" array.
Expected JSON Structure:
{"patchDocument": [
  // operators here
]}

Actual JSON Structure:
{"patchDocument": {
  "operations": [
    // operators here
  ]}
}

Hi @mehrzut , could you please provide swagger file example. expected and actual results for comparation?

Hi actually I found out that the swagger itself is showing the body like this:
  "patchDocument": {
    "contractResolver": {}
  }

but the backend developer says they used the .Net class as the input like this:
public JsonPatchDocument<Model> PatchDocument { get; set; }
Is there a problem with the backend?

That's would be great to take a look to full swagger file. you can rename everything and change urls if it's secret. Or share it to my email<EMAIL_ADDRESS>Because for now I don't know what is patchDocument and contractResolver :D
I want to check it in swagger editor and understand, what should be generated

I sent it to you.

@mehrzut yes, thanks, seen your message. Working on it

Hi @mehrzut , I've checked your swagger file. You can also check it in Swagger Editor. Looks like  patchDocument indeed the field of UpdateDriverCommand class. Example of request body looks like
{
  "driverId": 0,
  "patchDocument": {
    "contractResolver": {}
  }
}

For actual swagger file, according scheme, DriverJsonPatchDocument is a class with two properties: operations (List of DriverOperation) and contracResolver - IContractResolver class.
"DriverJsonPatchDocument": {
        "type": "object",
        "properties": {
          "operations": {
            "type": "array",
            "items": {
              "$ref": "#/components/schemas/DriverOperation"
            },
            "nullable": true,
            "readOnly": true
          },
          "contractResolver": {
            "$ref": "#/components/schemas/IContractResolver"
          }
        },
        "additionalProperties": false
      },

So, if this patchDocument is not needed - then it's swagger file issue. You can always check swagger file in Swagger Editor portal to check all schemas, requests and examples.
I'm closing the issue. But if you have any questions - feel free to reopen it.


scanning a file result in crash of elasticsearch and fatal error in malice

Describe the bug
every plugin result in failed to initalize elasticsearch
To Reproduce
simple pdf download on internet -> malice scan test.pdf
Expected behavior
result of scans displayed in the terminal
Environment (please complete the following information):
Output of docker version:
Client: Docker Engine - Community
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:47:43 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:55:00 2018
  OS/Arch:          linux/amd64
  Experimental:     true

Output of docker info:
Containers: 4
 Running: 1
 Paused: 0
 Stopped: 3
Images: 49
Server Version: 18.09.0
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.125-linuxkit
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952GiB
Name: linuxkit-025000000001
ID: BWXS:IEZB:NXVZ:5B4F:UCQH:OF3Z:P2HU:QYQD:EH6Z:3WPG:SPTA:U74X
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 39
 Goroutines: 64
 System Time: 2018-11-28T13:05:49.977260407Z
 EventsListeners: 2
HTTP Proxy: gateway.docker.internal:3128
HTTPS Proxy: gateway.docker.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false
Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, Docker For Mac, Docker Toolbox, docker-machine, etc.):
result of docker logs
[2018-11-28T13:03:37,969][WARN ][o.e.c.l.LogConfigurator  ] [unknown] Some logging configurations have %marker but don't have %node_name. We will automatically add %node_name to the pattern to ease the migration for users who customize log4j2.properties but will stop this behavior in 7.0. You should manually replace `%node_name` with `[%node_name]%marker ` in these locations:
  /usr/share/elasticsearch/config/log4j2.properties
[2018-11-28T13:03:38,200][INFO ][o.e.e.NodeEnvironment    ] [E2F-SSQ] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda1)]], net usable_space [38gb], net total_space [58.4gb], types [ext4]
[2018-11-28T13:03:38,200][INFO ][o.e.e.NodeEnvironment    ] [E2F-SSQ] heap size [990.7mb], compressed ordinary object pointers [true]
[2018-11-28T13:03:38,211][INFO ][o.e.n.Node               ] [E2F-SSQ] node name derived from node ID [E2F-SSQcSgip1JUxcYTW6g]; set [node.name] to override
[2018-11-28T13:03:38,211][INFO ][o.e.n.Node               ] [E2F-SSQ] version[6.5.0], pid[1], build[oss/tar/816e6f6/2018-11-09T18:58:36.352602Z], OS[Linux/4.9.125-linuxkit/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_181/25.181-b13]
[2018-11-28T13:03:38,211][INFO ][o.e.n.Node               ] [E2F-SSQ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/usr/share/elasticsearch/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.cgroups.hierarchy.override=/, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=tar]
[2018-11-28T13:03:38,902][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [aggs-matrix-stats]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [analysis-common]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [ingest-common]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [lang-expression]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [lang-mustache]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [lang-painless]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [mapper-extras]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [parent-join]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [percolator]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [rank-eval]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [reindex]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [repository-url]
[2018-11-28T13:03:38,903][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [transport-netty4]
[2018-11-28T13:03:38,904][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] loaded module [tribe]
[2018-11-28T13:03:38,904][INFO ][o.e.p.PluginsService     ] [E2F-SSQ] no plugins loaded
[2018-11-28T13:03:41,281][INFO ][o.e.d.DiscoveryModule    ] [E2F-SSQ] using discovery type [zen] and host providers [settings]
[2018-11-28T13:03:41,715][INFO ][o.e.n.Node               ] [E2F-SSQ] initialized
[2018-11-28T13:03:41,715][INFO ][o.e.n.Node               ] [E2F-SSQ] starting ...
[2018-11-28T13:03:41,877][INFO ][o.e.t.TransportService   ] [E2F-SSQ] publish_address {<IP_ADDRESS>:9300}, bound_addresses {<IP_ADDRESS>:9300}
[2018-11-28T13:03:41,894][INFO ][o.e.b.BootstrapChecks    ] [E2F-SSQ] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-11-28T13:03:44,935][INFO ][o.e.c.s.MasterService    ] [E2F-SSQ] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {E2F-SSQ}{E2F-SSQcSgip1JUxcYTW6g}{jxd5QjpPS4i15l1JbiuSjw}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}
[2018-11-28T13:03:44,938][INFO ][o.e.c.s.ClusterApplierService] [E2F-SSQ] new_master {E2F-SSQ}{E2F-SSQcSgip1JUxcYTW6g}{jxd5QjpPS4i15l1JbiuSjw}{<IP_ADDRESS>}{<IP_ADDRESS>:9300}, reason: apply cluster state (from master [master {E2F-SSQ}{E2F-SSQcSgip1JUxcYTW6g}{jxd5QjpPS4i15l1JbiuSjw}{<IP_ADDRESS>}{<IP_ADDRESS>:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2018-11-28T13:03:44,978][INFO ][o.e.h.n.Netty4HttpServerTransport] [E2F-SSQ] publish_address {<IP_ADDRESS>:9200}, bound_addresses {<IP_ADDRESS>:9200}
[2018-11-28T13:03:44,978][INFO ][o.e.n.Node               ] [E2F-SSQ] started
[2018-11-28T13:03:45,230][INFO ][o.e.g.GatewayService     ] [E2F-SSQ] recovered [3] indices into cluster_state
[2018-11-28T13:03:45,507][WARN ][r.suppressed             ] [E2F-SSQ] path: /malice/samples/IapfWmcBxXBSfaduVwcC, params: {index=malice, id=IapfWmcBxXBSfaduVwcC, type=samples}
org.elasticsearch.action.NoShardAvailableActionException: No shard available for [get [malice][samples][IapfWmcBxXBSfaduVwcC]: routing [null]]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.perform(TransportSingleShardAction.java:224) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.onFailure(TransportSingleShardAction.java:211) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.access$1200(TransportSingleShardAction.java:140) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction$2.handleException(TransportSingleShardAction.java:263) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1130) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1247) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1221) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:66) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.support.HandledTransportAction$ChannelActionListener.onFailure(HandledTransportAction.java:112) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$1.onFailure(TransportSingleShardAction.java:107) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:708) [elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) [elasticsearch-6.5.0.jar:6.5.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: org.elasticsearch.transport.RemoteTransportException: [E2F-SSQ][<IP_ADDRESS>:9300][indices:data/read/get[s]]
Caused by: org.elasticsearch.index.shard.IllegalIndexShardStateException: CurrentState[RECOVERING] operations only allowed when shard state is one of [POST_RECOVERY, STARTED]
	at org.elasticsearch.index.shard.IndexShard.readAllowed(IndexShard.java:1511) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.index.shard.IndexShard.get(IndexShard.java:891) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:165) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:87) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:79) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:87) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:43) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$1.doRun(TransportSingleShardAction.java:112) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:723) ~[elasticsearch-6.5.0.jar:6.5.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.5.0.jar:6.5.0]
	... 3 more
[2018-11-28T13:03:46,056][INFO ][o.e.c.r.a.AllocationService] [E2F-SSQ] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[malice][0]] ...]).
[2018-11-28T13:03:46,695][INFO ][o.e.c.m.MetaDataMappingService] [E2F-SSQ] [malice/ayNdUZsZQkedL6EtHbDDZQ] update_mapping [samples]

Additional context
Add any other context about the problem here.

Did you try these? https://github.com/maliceio/malice/blob/master/docs/KnownBugs.md

Since it was on Macos (I know Linux base), I wasn't sure and I wanted your views first. I'm going to test it.

De : blacktop<EMAIL_ADDRESS>Envoyé : mercredi, novembre 28, 2018 2:13 PM
À : maliceio/malice
Cc : Impostor syndrom; Author
Objet : Re: [maliceio/malice] scanning a file result in crash of elasticsearch and fatal error in malice (#82)
Did you try these? https://github.com/maliceio/malice/blob/master/docs/KnownBugs.md
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/maliceio/malice/issues/82#issuecomment-442441970, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJFPg0tWUYcGQDr851aB8eer5YCW-s9Bks5uzovsgaJpZM4Y3kRd.

okey by updating the VM to 4G, I got some results.
and it seems that the process can go through the whole protocol without crashing.
I have still a lot of failed to update sample in a lot of plugin. Should I do a bug report for that or is it normal?
failed: Yara, mcafee, fprot, escan, clamav, comodo, fsecure
succeeded: zoner, avast, dr web, bitdefender, sophos.
It stays stuck after the result of sophos but the container didn't crash...

6G of RAM for a 30MB pdf. Don't know if the size of the file has anything to do with it but so you know.
still a 409 error with virustotal
time="2018-11-29T00:45:02Z" level=fatal msg="failed to index malice/virustotal results: failed to update sample with id: RYrtXGcBjIpm6xqHErD5: elastic: Error 409 (Conflict): [samples][RYrtXGcBjIpm6xqHErD5]: version conflict, current version [2] is different than the one provided [1] [type=version_conflict_engine_exception]" category=intel hash= plugin=virustotal


add redux icon

Is the logo any different to #139 ?

Sorry, I had not noticed.
then I close it since it is duplicated.

Okay, no worries.


idset.go: micro-optimise away redundant scan

I saw this small thing when studying the code.

This is a nice finding, thank you!


New policy: Disallow Duplicate Deployment ServiceAccount

Description
Policy to check if every Deployment is using a unique service account, and rejects if the deployment's service account is already in use by a existing deployment.
Examples
Given the following Deployment is already defined:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: default
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      serviceAccountName: nginx
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

The creation of the following Deployment is going to be accepted:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress-deployment
  namespace: default
  labels:
    app: wordpress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wordpress
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      serviceAccountName: wordpress
      containers:
      - name: wordpress
        image: wordpress:latest
        ports:
        - containerPort: 80

The creation is allowed because the serviceAccountName is different.
The creation of the following Deployment is not going to be allowed:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress-deployment
  namespace: default
  labels:
    app: wordpress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wordpress
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      serviceAccountName: nginx
      containers:
      - name: wordpress
        image: wordpress:latest
        ports:
        - containerPort: 80

The creation is rejected because the nginx service account is already in use by the nginx-deployment resource.
The creation of the following Deployment is going to be accepted:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress-deployment
  namespace: internal
  labels:
    app: wordpress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wordpress
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      serviceAccountName: nginx
      containers:
      - name: wordpress
        image: wordpress:latest
        ports:
        - containerPort: 80

The creation is allowed despite the workload using the nginx service account. That happens because this deployment is done inside of a different Namespace (staging vs default).
Notes
This is going to be a context aware policy that requires access to the Deployment resources

@Martin-Weiss, @raif-ahmed can you check the description of the policy to ensure it matches your requirements?

@flavio Yes this is correct.

@flavio  - do we know if the applications we deliver with rancher already follow this rule (not to use the same service account for more than one deployment / daemon set / stateful set / jobs?
And - would this policy not be required for all sets of "deployment / daemon set / stateful set / jobs"?
One more thing - we might need a policy that denies the usage of the "default service account in any namespace" forcing applications to create their own, always... (in case someone does not specify a service account in the "deployment / daemon set / stateful set / jobs" at all)

@flavio - do we know if the applications we deliver with rancher already follow this rule (not to use the same service account for more than one deployment / daemon set / stateful set / jobs?

I don't know about that. We can deploy this policy in monitor mode and then rely on the audit scanner report to identify all the workloads that are violating it.

And - would this policy not be required for all sets of "deployment / daemon set / stateful set / jobs"?

Sorry, I didn't understand it. Can you elaborate a bit on that?

One more thing - we might need a policy that denies the usage of the "default service account in any namespace" forcing applications to create their own, always... (in case someone does not specify a service account in the "deployment / daemon set / stateful set / jobs" at all)

Is using the default service account considered a bad security practice?
According to the official docs, the privileges given to it are really really thin:

The default service accounts in each namespace get no permissions by default other than the default API discovery permissions that Kubernetes grants to all authenticated principals if role-based access control (RBAC) is enabled.

If using the default service account is not considered a bad security practice, we should ensure this policy doesn't enforce the uniqueness of it. Meaning, any number of workloads can use the default Service Account

@flavio - do we know if the applications we deliver with rancher already follow this rule (not to use the same service account for more than one deployment / daemon set / stateful set / jobs?

I don't know about that. We can deploy this policy in monitor mode and then rely on the audit scanner report to identify all the workloads that are violating it.

Maybe I misundestood as well - does the policy just look at deployments or at pods or even statefulset/daemonset/job as well?
I understood that this policy seems to verify "service account in deployment A vs. service account in deployment B" - but it could also be "service account in deployment A vs. service account in statefulset C".
-> so checking for "duplicate / multiple usage of the same service account might need to be done cross-<deployment/daemon-set/job/statefulset/pod> and not just vs

And - would this policy not be required for all sets of "deployment / daemon set / stateful set / jobs"?

Sorry, I didn't understand it. Can you elaborate a bit on that?

See above.

One more thing - we might need a policy that denies the usage of the "default service account in any namespace" forcing applications to create their own, always... (in case someone does not specify a service account in the "deployment / daemon set / stateful set / jobs" at all)

Is using the default service account considered a bad security practice?
According to the official docs, the privileges given to it are really really thin:

Similar to using the "default" namespace I would see "using separe service accounts" as a security recommendation similar to "do not share security accounts".
This might be relevant from an auditing point of view - which process used which account do do x,y,z..?
And this might be relevant from a security point of view - for pod A we give the default service account a rolebinding for "much more rights" and then pod B is using / getting this as well..
So a policy that denies the usage of the default service account might also improve security (but not sure what this will break :-))

The default service accounts in each namespace get no permissions by default other than the default API discovery permissions that Kubernetes grants to all authenticated principals if role-based access control (RBAC) is enabled.

If using the default service account is not considered a bad security practice, we should ensure this policy doesn't enforce the uniqueness of it. Meaning, any number of workloads can use the default Service Account

What is "best" practice ;-) and what is the attach surface.. - IMO using a default user in general is not a good practice these days.. (root / administrator) ...

Maybe I misundestood as well - does the policy just look at deployments or at pods or even statefulset/daemonset/job as well?
I understood that this policy seems to verify "service account in deployment A vs. service account in deployment B" - but it could also be "service account in deployment A vs. service account in statefulset C".

You're right. The check was not going to be done across different Kubernetes types. I'll update the card.

Similar to using the "default" namespace I would see "using separe service accounts" as a security recommendation similar to "do not share security accounts".
This might be relevant from an auditing point of view - which process used which account do do x,y,z..?
And this might be relevant from a security point of view - for pod A we give the default service account a rolebinding for "much more rights" and then pod B is using / getting this as well..
So a policy that denies the usage of the default service account might also improve security (but not sure what this will break :-))

I'm afraid many things would break by denying the usage of the default Service Account. We can leave that as an optional setting. Something like:
settings:
  deny_usage_of_default_service_account: true # false by default

Thanks - and yes - I am sure that many things will break when we deny usage of default service account.
From a security point if view I still think that "personalized service accounts" should be better than "default" - especially when looking into auditing... (bumpy road in front of us ;-) ) - but let users decide and I agree with the default "false"..


Options not shown in --help

I just discovered yeoman-option-or-prompt and really like the idea.
But it would be nice if the options would be shown when the generator is called with --help. Any chance?

@PhilippSoehnlein that's up to the author of the generator. This project just provides a function that allows generators to use options to override prompts. The generator author should decide whether to publish those options as part of the generator's interface. See http://yeoman.io/authoring/user-interactions.html for details on how to declare options. I assume that declaring options in the constructor is what allows yo to show them with yo <generator> --help.
If you are using yeoman-option-or-prompt, you can just declare options in your generator's constructor that match the names of the prompts that you're passing to optionOrPrompt.