Generation failed for OAS containing a multipart with a string binary property

Description
We tried to generate python code from an OAS containing a multipart with a string binary property.
Using the same example but with modelerfour v4.21.0 made it work.
Error
AutoRest code generation utility [cli version: 3.6.2; node: v16.18.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    |    Loading AutoRest core      '/root/.autorest/@autorestcore@3.9.1/nodemodules/@autorest/core/dist' (3.9.1)
info    |    Loading AutoRest extension '@autorest/python' (6.1.1->6.1.1)
info    |    Loading AutoRest extension '@autorest/modelerfour' (4.23.7->4.23.7)
warning | PreCheck/BinarySchema | 'The schema post-content-schema-image with 'type: string', format: binary' will be treated as a binary blob for binary media types.
    - file:///mnt/specs/openapi.yml:22:17
warning | PreCheck/CheckDuplicateSchemas | Checking for duplicate schemas, this could take a (long) while.  Run with --verbose for more detail.
warning | Modeler/Superflous type information | 'The schema post-content-schema-image with 'type: string', format: binary' will be treated as a binary blob for binary media types.
error   | [_main_.Process:75] Python generator raised an exception
Traceback (most recent call last):
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 73, in Process
    return plugin.process()
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 105, in process
    self.updateyaml(yamldata)
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 1120, in updateyaml
    yaml_data["operationGroups"] = [
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 1121, in <listcomp>
    self.updateoperation_group(og) for og in yamldata["operationGroups"]
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 650, in update_operationgroup
    [
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 651, in <listcomp>
    self.getoperation_creator(o)(propertyname, o)
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 555, in updateoperation
    self.updatebody_parameter(yamldata["requestMediaTypes"])
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 719, in update_bodyparameter
    return self.updatemultipart_bodyparameter(
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 698, in update_multipart_bodyparameter
    entries = [
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 699, in <listcomp>
    self.update_body_parameter_helper(yaml_data, p, updatetype(p["schema"]))
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 667, in _update_body_parameterhelper
    parambase = update_parameter_base(bodyparam)
  File "/root/.autorest/@autorestpython<EMAIL_ADDRESS>line 304, in update_parameterbase
    "description": yaml_data["language"]["default"]["description"],
KeyError: 'description'
fatal   | Process() cancelled due to failure
error   |   Error: Plugin m4reformatter reported failure.
error   | Autorest completed with an error. If you think the error message is unclear, or is a bug, please declare an issues at https://github.com/Azure/autorest/issues with the error message you are seeing.

Configuration
python: true
black: false
version: 3.9.1
use-extension:
  '@autorest/python': 6.1.1
  '@autorest/modelerfour': 4.23.7

Trimmed Example
openapi: 3.0.3

info:
  description: ""
  version: 1.0.0
  title: Example Multipart form with binary

paths:
  /image:
    post:
      description: Post an image
      operationId: addImage
      responses:
        '200':
          description: OK
      requestBody:
        content:
          multipart/form-data:
            schema:
              type: object
              properties:
                image:
                  type: string
                  format: binary

thank you for reporting this issue, @changlong-liu @msyyc can one of you take a look today? Thanks!

@tylercopeland31 I also see you're using an outdated version of autorest.python, can you try not specifying the version when you generate? Generating with --python will get you the latest version (i believe 6.2.6). Does the issue still persist?

Hi @changlong-liu , setting the description as you suggested we get the next warning:
warning | [autorest.codegen.models.basebuilder.__init__:68] Not going to generate operation "build_add_image_request" because we are unable to generate this type of operation right now. Please write your own custom operation in the "patch.py" file following https://aka.ms/azsdk/python/dpcodegen/python/customize
warning | [autorest.codegen.models.basebuilder.__init__:68] Not going to generate operation "add_image" because we are unable to generate this type of operation right now. Please write your own custom operation in the "patch.py" file following https://aka.ms/azsdk/python/dpcodegen/python/customize

Is something there that can I do to fix it without touching the "patch.py" file?

Hi @changlong-liu , setting the description as you suggested we get the next warning:
warning | [autorest.codegen.models.basebuilder.__init__:68] Not going to generate operation "build_add_image_request" because we are unable to generate this type of operation right now. Please write your own custom operation in the "patch.py" file following https://aka.ms/azsdk/python/dpcodegen/python/customize
warning | [autorest.codegen.models.basebuilder.__init__:68] Not going to generate operation "add_image" because we are unable to generate this type of operation right now. Please write your own custom operation in the "patch.py" file following https://aka.ms/azsdk/python/dpcodegen/python/customize

Is something there that can I do to fix it without touching the "patch.py" file?

Hi @jrpedrianes , currently the patch.py is the only way to work with '--version-tolerant' generation of multipart/form-data. Will this sample file help you on writing it? https://github.com/Azure/autorest.python/blob/autorestv3/packages/autorest.python/test/vanilla/version-tolerant/Expected/AcceptanceTests/BodyFormDataVersionTolerant/bodyformdataversiontolerant/operations/_patch.py

Thanks, adding the descriptions and creating a _patch.py works for me


PEP8 style sweep

Without affecting test ratios, I have made a sweep of the Arrays
algorithms, and improved style, commentary, whitespace, and some
translations
flatten.py -- added commentary
Garage.py -- added commentary, enforced line width on existing
commentary
Josephus.py -- enforced commentary alignment, binary operator spacing
Limit.py -- enforced binary operator spacing, enforced commentary width
This function is very similar to Python array-slicing, except by value.
longest_non_repeat.py -- enforced spacing style in commentary
added get functions
Enforced function whitespace of one carriage return
merge_intervals.py -- switched docstrings so the subject exposition was
in the intro and the action done by the class was under the class,
cleaned up other docstrings
-- this class does need a docstring, but not the mathematician's
exposition. What does this object do?
n_sum.py -- added whitespace around recursive call, added (TL:DR)
commentary, added docstrings, added commentary identifying recursive
call, enforced line width and pep8 line break style WRT parameters on
newlines w/ parentheses (e.g. lines 81-84, 88-90), enforced spelling
consistency in argument parameters (e.g. target/taget)
plus_one.py -- removed unused variable in plus_one_v3 method, clarified
opening docstring
rotate.py -- added comment line 24 for intentionally unused variable,
enforced whitespace guidelines between methods
three_sum.py -- specified opening docstring to indicate distinct
elements in solution sets
top_1.py -- improved translation in commentary
trimmean.py -- improved translation in commentary, removed leftover
commented instrumentation

Included language inviting contributors to improve documentation.
(Put an X inside the [ ] to denote check mark [X].)

[ ] If creating a new file :

[ ] added links to it in the README files ?
[ ] included tests with it ?
[ ] added description (overview of algorithm, time and space compleixty, and possible edge case) in docstrings ?

[X] if done some changes :

[X] wrote short description in the PR explaining what the changes do ?
[X] Fixes #[12], #[242] if related to any issue

[x] other

Thanks for the contribution!
I gave a brief look, but it looks good to me.
I am adding some minor comments and come back and do a full review today or tomorrow.

ill merge this PR since it is not affecting logic


Tidy and Critic should not check pod files

@jrha ideally, this gets merge and a release is made (for maven; making an rpm as part of the release is not needed atm). i'm testing ncm-ncd atm, it's very useful

@ned21 we have quite a bit code that does not use Test::Quattor to do actual non-mocked testing (in particular, CAF can't use it). but it should always be non-disruptive to import Test::Quattor::Object in existing tests to enable the warning test.

@stdweird thanks for the explanation. Apologies I actually seem to have copy & pasted the wrong commit id. It was 85c9e66 (critic and tidy codedirs default to doc poddirs) that I didn't understand.

as we have most of the pod either in the scripts and/or modules, or have pod file in the directory as the code; the checks typically scan the same directories for code to check. so instead of configuring it 3 times with the same value, we can do so using one value. the doc directory is chosne because we currently have pod tests with the list of dirs.
eg https://github.com/quattor/maven-tools/blob/master/build-scripts/src/test/perl/00-tqu.t#L12
i should remove the codedirs value in the critic and tidy section.

OK. Since 16.12 is done and @jrha is on holiday, should this get merged or wait for @jrha ?

@ned21 i don't think 16.12 is out, but maven-tools doesn't follow the releases anyway
i also pushed and squashed a commit fixing the redundant codedir entries in the 00-tqu unittest.


Entity position doesn't apply to polygons

This seems like such a fundamental thing that it must be an intended behavior, but I'm bringing it up here because I feel like it's confusing and I haven't found any issues discussing it.
Let's say you have a polygon entity:
var redPolygon = viewer.entities.add({
    polygon : {
        hierarchy : Cesium.Cartesian3.fromDegreesArray([-115.0, 37.0,
                                                        -115.0, 32.0,
                                                        -107.0, 33.0,
                                                        -102.0, 31.0,
                                                        -102.0, 35.0]),
        material : Cesium.Color.RED
    }
});

Looks great. Let's say you wanted to move it just slightly, so you define a position offset:
var redPolygon = viewer.entities.add({
    position : new Cesium.Cartesian3(100, 0, 0),
    polygon : {
        hierarchy : Cesium.Cartesian3.fromDegreesArray([-115.0, 37.0,
                                                        -115.0, 32.0,
                                                        -107.0, 33.0,
                                                        -102.0, 31.0,
                                                        -102.0, 35.0]),
        material : Cesium.Color.RED
    }
});

This has no effect. This is confusing because the docs imply that this is the position of the entity, so the fact that it doesn't change anything is strange. This seems to be the case with other geometry types but not with models.
Of course it's possible to just offset every positions in polygon.hierarchy, but this seems like an inconsistent API. Why is position a property of the entity if it only applies to model ?
The fact that entities don't have a model matrix makes it so that there's no easy way to apply offsets like this. This came up while I was trying to show that classification entities can be dynamic, and with invert classification you could do a cool thing where you can have a moving mask over your tileset. So it would have been nice to be able to just specify a position and move that rectangle anywhere.
I know this would be kind of a big API change, but at the very least something in the docs should say that entity.position doesn't apply to all geometry types.

@OmarShehata position applies to certain entity types like labels, billboards, models, ellipses, pretty much any entity type that has a singular position.  It does not apply to geometry types with multiple positions like polygons, polylines and walls.  I think we should just clarify the documentation, I don't think this is a change we would want to make in the near future.  Applying a modelMatrix to a geometry type that is intended to curve to the surface of the globe does not have good results.  The best way to move the polygon would be to make the hierarchy a dynamic property and recalculate the position values.

@hpinkos your suggestion to move with dynamic property not working well.
background:
In my project, I work with angular js,
I want to move a polygon on a map by locations objects that change by indexes.
I tried to move a polygon by accesses to  poly.polygon.hierarchy / poly["polygon"]["hierarchy"]  but a wired thing happened.
when i change the index automaticlly the polygon not move at all.
just after I stop the index the polygon updated to the correct location.
There is no any asynchronous problem in my code.
When I debug I see that  poly.polygon.hierarchy change the place correctly but it's not rendered until I stop the index running
//vm.fov locations objects that change
`$scope.$watch( 'vm.fov', function(newIndex,oldName){
if(vm.fov !== "" && carModel){
moveCar();
}
});
//moving car and fov -  tirnagle polygon
function moveCar(){
var position =  getCurrentPosition() ;
var orientation = getOrientation (position );
carModel.position = position;
carModel.orientation = orientation;
carFov.polygon.hierarchy = new Cesium.PolygonHierarchy(
Cesium.Cartesian3.fromDegreesArray([
vm.fov["position"].lon, vm.fov["position"].lat,
vm.fov["left"].lon,vm.fov["left"].lat,
vm.fov["right"].lon,vm.fov["right"].lat
         ]));

}`

where am I wrong / is this bug?

@ezriYerushalmi this is a common issue. CesiumJS assumes most properties are static for performance. I think if you use a Callback property to mark it as dynamic it should work. See this example:
https://cesiumjs.org/Cesium/Build/Apps/Sandcastle/?src=Callback Property.html

@ezriYerushalmi here's an example of using a CallbackProperty for retrieving the polygon hierarchy
var viewer = new Cesium.Viewer('cesiumContainer');

var i = 0;
function getHierarchy() {
    return Cesium.Cartesian3.fromDegreesArray([-115.0 + i, 37.0,
                                        -115.0 - i, 32.0,
                                        -107.0, 33.0,
                                        -102.0, 31.0,
                                        -102.0, 35.0]);
}

setInterval(function() {
    i += 0.01;
}, 50);

var redPolygon = viewer.entities.add({
    name : 'Red polygon on surface',
    polygon : {
        hierarchy : new Cesium.CallbackProperty(getHierarchy, false),
        material : Cesium.Color.RED
    }
});

viewer.zoomTo(viewer.entities);

Works great thank you all!
@OmarShehata @hpinkos


Become a Collaborator

Hi All, If you have ever landed non-trivial contributions before, you can comment on this issue so as to apply for being a collaborator. Other collaborators will give thumbs up or thumbs down to your contribution, and we will add you to the collaborators list if you have enough supporters.

啥意思？ 后期改为商业的？


Support for Govee Music Sync Box (H1162011)

I'm not sure if Govee's Music Sync Box (H1162011) is supported by the API. If it is, it would be great to have it available as a switch (ON/OFF)  in HomeKit.
If it's not, understood and this can be closed.

hi @GeorgeeNico do you own this model already?

I do.

do you have the plugin running? please enter your username and password in the config, you should see a message in the log about this device not being supported, would be useful to have this log message.

Yup, here you go:
[21/01/2023, 14:42:22] [Govee] [Govee Music Sync Box] is not currently supported, do create a Github issue with the following info: {"device":"9B:66:D0:32:38:30:2F:6A","deviceName":"Govee Music Sync Box","model":"H1162","httpInfo":{"groupId":0,"device":"9B:66:D0:32:38:30:2F:6A","sku":"H1162","spec":"","versionHard":"3.01.00","versionSoft":"1.01.03","deviceName":"Govee Music Sync Box","deviceExt":{"deviceSettings":"{\"address\":\"D0:32:38:30:2F:6A\",\"bleName\":\"GVH11622F6A\",\"topic\":\"GD/27beebf4e2b1a4f26511c0071435d23b\",\"pactType\":1,\"pactCode\":1,\"wifiSoftVersion\":\"\",\"wifiHardVersion\":\"\",\"ic\":0,\"secretCode\":\"NzVjNDFkMTE=\",\"boilWaterCompletedNotiOnOff\":1,\"sku\":\"H1162\",\"device\":\"9B:66:D0:32:38:30:2F:6A\",\"deviceName\":\"Govee Music Sync Box\",\"versionHard\":\"3.01.00\",\"versionSoft\":\"1.01.03\",\"playState\":false}","lastDeviceData":"{\"online\":false}","extResources":"{\"skuUrl\":\"https://d1f2504ijhdyjw.cloudfront.net/sku-img/5f8b337fec9c0c7b321e13699a0d57f8-add_list_type_device_1162.png\",\"headOnImg\":\"https://d1f2504ijhdyjw.cloudfront.net/sku-img/eaa00e28db07555a4b95a4913c54dcce-new_switch_pics_h1162_on%403x.png\",\"headOffImg\":\"https://d1f2504ijhdyjw.cloudfront.net/sku-img/e3411556ccf090b87205de604b1b2dd7-new_switch_pics_h1162_off%403x.png\",\"ext\":\"\",\"ic\":0}"},"pactType":1,"pactCode":1,"goodsType":87}}

please update to the beta version of the plugin. this will be added to the home app as an "unsupported accessory".
However you should see logging appear in the homebridge log when you use the govee app to switch this device on and off (please disable bluetooth on your phone to force the device to connect via wifi).
If you should show me the logs for switching the device ON and OFF I should be able to implement this.

Done, though I'm starting to realize this product likely does not have wifi... It may work only through bluetooth...
[22/01/2023, 18:28:42] [Govee] [Govee Music Sync Box] initialising with options {}.
[22/01/2023, 18:28:42] [Govee] [Govee Music Sync Box] initialised with id [9B:66:D0:32:38:30:2F:6A] [H1162].

Nothing in the log comes up related to the device, may not be possible to build into the plugin.

which platform are you running homebridge on?
(if a raspberry pi) how close is this device physically? and would you consider yourself pretty technical? i know the last question is quite relative 😁

Running on a raspberry pi 4 and it's close enough to reach the device.
Problem is that I've been having trouble with plugins being able to access to the BLE connection, I gave up on this a while ago and never bothered to fix it (I have the same problem with the SwitchBot plugin):
[23/01/2023, 13:24:43] [Govee] [BLE] disabling client as ERR_DLOPEN_FAILED.
I'm technical enough and am willing to try and figure this out with you, though it may be slow to get there (especially with the BLE issue I've been facing).


Not able to load images from Amazon s3 bucket

Hi there,
FastImage is not able to load image from Amazon S3 bucket. The image is public and ContentType image/jpeg.
Does anyone faced this issue before? I would appreciate if someone can help me with a workaround.
Thanks.

I am loading loads of images from S3 without any issues. Can you give more information? A link to one of those images could be helpful!

Hi @n1ru4l
https://s3.eu-central-1.amazonaws.com/megroplan-test/recipe/1528058084_BuNpnvWM44.jpeg
The image is displayed using  from React Native.
I have a Flatlist and below is my render item:
`renderItem(item){
return ( <FastImage
style={styles.rowItemRecipeImageContainer}
source={{uri: item.image,
priority: FastImage.priority.high,
}}
resizeMode={FastImage.resizeMode.contain}
>
                <View style={styles.rowItemRecipeBottomSection}>
                    <View style={styles.rowItemRecipeBottomSectionLeft}>
                        <Text numberOfLines={1}  style={styles.rowItemRecipeTitle}>{item.title}</Text>
                    </View>
                    <View   style={styles.rowItemRecipeBottomSectionArrow}>
                        <Icons color={'white'} size={32} onPress={ () => {} } name={"navigate-next"}/>
                    </View>
                </View>

            </FastImage>
);

}
`

@bfdevelop Are you able to load other images, beside those from your s3 bucket?

@n1ru4l  Yes, I can load other images, even large images.
You did not use any special properties, right?

@bfdevelop  Did you resolve the issue? If so please comment your solution for other users that experience this issue!

Someone else had an issue similar to this and they found they had some kind of user agent check preventing the image from loading.

I have the same issue the images not load for aws services. is there any solution..?

I too have the same issue, I am passing sample image uri's in the fast-image tag and rendering it. But no Images are displayed.
Below is the code:
import React, {Component} from 'react';
import {Platform, StyleSheet, Text, View} from 'react-native';
import FastImage from 'react-native-fast-image'

const YourImage = () => (
  console.log("this is called"),
  <FastImage
      style={styles.image}
      source={{
          uri: "https://luehangs.site/pic-chat-app-images/animals-avian-beach-760984.jpg",
          headers: { Authorization: 'someAuthToken' },
          priority: FastImage.priority.normal,
      }}
      resizeMode={FastImage.resizeMode.contain}
  />
)

export default YourImage;

I am not loading any images from S3. All i am doing is passing a sample image Uri, but all i see is a blank screen.

I am not loading any images from S3. All i am doing is passing a sample image Uri, but all i see is a blank screen.

You need remove this string
 headers: { Authorization: 'someAuthToken' }
This work for me.

Any solution here?

any updates?

See https://github.com/DylanVann/react-native-fast-image/issues/242#issuecomment-638732002 for some ideas on things to try.

If your backend is validating user-agent for the authorization token, check if the user-agent used for generating auth token is same for fast image as well.
In my case setting the right user-agent fixed the problem.
import { getUserAgent, getUserAgentSync } from 'react-native-device-info';
const source = {
   uri: 'http://yoururl',
    headers: {
        Authorization: `Bearer ${TokenService.getValidAccessTokenSync()}`,
        'user-agent': getUserAgentSync()
      },
}

Please try this format may be it works
<FastImage
key={idx}
src={{
uri: 'http://yoururl' +  '?noCache=' + Math.random().toString(),
method: 'GET',
headers: { 'Access-Control-Allow-Origin': '*'},
body: ''
}}
/>

Please try this format may be it works
<FastImage
key={idx}
src={{
uri: 'http://yoururl' + '?noCache=' + Math.random().toString(),
method: 'GET',
headers: { 'Access-Control-Allow-Origin': '*'},
body: ''
}}
/>

Work's for me when i added method: 'GET',  to source params


add SoundSystem.stop_channel

with this, we can have a pretty complete basic sound system for simple video games. Is it enough for 0.1.1 or are there stil missing features?

Looks good to me, do you think we need pause_channel ?

I don't see a concrete use case for that since uni_snd is already pausing the sound system when the browser tab is hidden. Do you have something in mind ?

No, just wonder if it is useful and i agreed with you, i will merge it now.


[FIX] File name is longer than the maximum allowed path length in some OS

exchange positiion checking isFilePath and isBase64 and catch the error while checking is_file

Thanks.


Add customFieldValue method to Context

This PR adds the customFieldValue method to the Context class which gets a particular custom field value and parses it into it's appropriate type.

Current dependencies on/for this PR:

main

PR #34

PR #35   👈

PR #36

This comment was auto-generated by Graphite.


Stack trace is missing due to bootstrap.php.cache

In sentry i dont see a full stack trace cause of bootstrap.php.cache, is there a configuration i am missing?

Can you post an example of what you are missing?
Why do you think that the boostrap has merit here?
Also, part of the stack trace is suppressed if it's outside of your app.

Closing this since no further info were provided. Feel free to reply if you have them.


Do not pass all host devices in /dev when launching with '--privileged'

Description of problem
When i launch an android image with --privileged parameter, all device under host /dev directory will be passed into the container directory /dev, the behavior doesn't make sense;
Expected result
Don't pass all host devices into kata container even with --privileged parameter;
Actual result
Here is thels info under kata container directory /dev:
root@5ff9d5d81457:/dev# ls
agpgart          dri    input         lp3                 null    raw   shm       tap21605  tty14  tty21  tty29  tty36  tty43  tty50  tty58  tty8    urandom  vcs6   vfio
autofs           fb0    kmsg          mapper              nvram   rtc0  snapshot  tty       tty15  tty22  tty3   tty37  tty44  tty51  tty59  tty9    usbmon0  vcsa   vga_arbiter
bsg              fd     kvm           mcelog              oldmem  sda   snd       tty0      tty16  tty23  tty30  tty38  tty45  tty52  tty6   ttyS0   vcs      vcsa1  vhci
btrfs-control    fd0    loop-control  mem                 port    sda1  sr0       tty1      tty17  tty24  tty31  tty39  tty46  tty53  tty60  ttyS1   vcs1     vcsa2  vhost-net
console          full   loop0         mqueue              ppp     sda2  stderr    tty10     tty18  tty25  tty32  tty4   tty47  tty54  tty61  ttyS2   vcs2     vcsa3  vmci
cpu              fuse   lp0           net                 ptmx    sda3  stdin     tty11     tty19  tty26  tty33  tty40  tty48  tty55  tty62  ttyS3   vcs3     vcsa4  vsock
cpu_dma_latency  hpet   lp1           network_latency     pts     sg0   stdout    tty12     tty2   tty27  tty34  tty41  tty49  tty56  tty63  uhid    vcs4     vcsa5  zero
crash            hwrng  lp2           network_throughput  random  sg1   tap20463  tty13     tty20  tty28  tty35  tty42  tty5   tty57  tty7   uinput  vcs5     vcsa6
root@5ff9d5d81457:/dev#

Here is the host view of docker ps:
[root@centos-k8s014 home]# docker ps
CONTAINER ID        IMAGE                                COMMAND             CREATED             STATUS              PORTS               NAMES
5ff9d5d81457        sgjesse/legacy-android-base:latest   "/bin/bash"         About an hour ago   Up About an hour                        gracious_wright
619d72f2ad28        android:NAe000047                    "/android-entry"    About an hour ago   Up About an hour                        friendly_ritchie
822fa322b90c        android:NAe000047                    "/android-entry"    About an hour ago   Up About an hour                        dazzling_fermi

Please see the related issue and root cause here:
https://github.com/kata-containers/runtime/issues/1551

any idea about this issue?

Hi @zhiminghufighting - I've updated the description and reformatted the details to make this easier to understand. Please can you provide details of which devices you would not expect to see?

Related: https://github.com/kata-containers/runtime/pull/1558.
/cc @amshinde.

@zhiminghufighting I am not sure why you are seeing this behaviour,  before I go take a look, can you tell me what is the specific android image that you are using?

This is highly unexpected and undocumented behavior that just fell on my feet during a CTF competition that uses Kata Containers for isolation :)
I passed --privileged=true such that I could run Docker containers inside the VM, but this also resulted in full host access that some players used to break out of the VMs.
Applied a local hotfix (no-ip filterDevices), but I suggest the default behavior be changed.

@leoluk - saw your PR over at https://github.com/kata-containers/documentation/pull/452, thx.
Heh, I guess it is a question of 'when is --privileged not really --privileged... I have a feeling we may never please all the users all the time on this one ;-)
/cc @amshinde for thoughts and input.

@amshinde  I use a android based container which is running in Tencent cloud gaming.(This android based image also is made by intel AIS team for Tencent cloud gaming)
I tried in many conditions and found it only can be found in this condition : Host - Ubuntu + Kata + android based container;
If the Host OS is changed to Centos(7.3), it will never happen. i am not sure if it is related with OS policy?
By the way, the Ubuntu OS is 18.04; Kata is 1.6; i haven't tried on other combination.
@jodh-intel Thanks for your comment and corrections.

@zhiminghufighting Sorry for the delay in getting back.
Can you run a ls -l  /dev on the host and inside the container so that we can compare the devices that are passed to the container. We should be passing only block and vfio devices from the host to the guest and not all devices. It is not possible to pass all host devices to a virtual machine.
Finally, are you running this under docker or k8s? Could you provide the config.json file that gets passed to the runtime. For docker this should be under /var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby{containerid}/config.json
If you could point me to the android image that I should use, I can try and reproduce this as well.

the number os devices are different in the host, over kata and over runc
iuri@evolux-clx:~$ docker run --runtime=runc --privileged --rm alpine ls /dev -1 | wc -l
229
iuri@evolux-clx:~$ docker run --runtime=kata --privileged --rm alpine ls /dev -1 | wc -l
228
iuri@evolux-clx:~$ sudo ls /dev -1 | wc -l
[sudo] password for iuri:
242
iuri@evolux-clx:~$

I didn't get, what do make you to think that devices are shared in --privileged mode? which is the evidence?

Can we make this configurable, at least (and default it to off)? There's plenty of use cases that need privileged access inside the container only.

Hi @leoluk, can you share a bit more about how you do filterDevices as the hotfix?

+1 for this issue. Using Kata to isolate and secure Docker in docker is pointless when all of the host devices are simply host plugged into the VM, including the host's / device. You may as well just do away with the VM and run it as a regular runc container.
I would like to see a configurable option in the Kata config to control filtering of the /dev devices, that way when enable the /dev devices are filtered out. Or maybe a configurable regex and pass the device name through?
Another possible option that @egernst suggested is to add support for an annotation, that when used would enable a more kata specific privileged mode - support for all capabilities, writable sysfs, but no host devices. A cut down version of --privileged.
Thoughts?
cc @egernst @amshinde @ganeshmaharaj @mcastelino

-1.
--privileged in docker translates into two things:

pass all devices under /dev to container
give container all sys capabilites

If you just need the second one, you can list the required capabilities you need with docker run --cap-add. --privileged is just a lazy and dumb shortcut. docker does provide precise options. --cap-add and --device are your friends if you want precise control.
OTOH, if users require a simple way to specify that my container needs all sys capabilites but no extra implicit devices, please file issues in the moby project to request such a flag. It is not kata's role to workaround the issue.

@bergwolf

--privileged in docker translates into two things:

pass all devices under /dev to container
give container all sys capabilites

No, it translates into a bit more than that, see what is provided in containerd - https://github.com/containerd/containerd/blob/172fe90e55c3c3a452f9bec926d87ffda5ed01bf/oci/spec_opts.go#L1090-L1101

@awprice Still, please file issues in moby/containerd to request for such a flag that does not send all /dev devices to the OCI runtime. It is not kata's role to ignore random part of the container spec. E.g., if we implement what's requested here, there is no way to pass a device to a container when --privileged is specified. What's the point of supporting one specific use case while breaking others?
Generally --privileged is broken in many ways for kata and I would strongly recommend against using it. E.g., what's the point of passing host /dev/fuse to the guest? OTOH, it is improper to ignore a container's device list all together. The problem is that we(kata runtime) actually don't know which device is implicitly added and which is specified by users. IOW, you are fixing the problem in a wrong layer.
At very least, I'm fine to use an annotation as @egernst suggested to tell that users actually want to ignore all devices. At least we know that users do want it and it is not a wild guess on the real use case. But still I think moby/containerd is the right place for the fix.

Kata is already interpreting the container spec quote liberally due to the extra abstraction layer involved. For instance, capabilities will not allow you to access the host system, but only the guest kernel in the VM. Why make an exception for devices?

E.g., if we implement what's requested here, there is no way to pass a device to a container when --privileged is specified. What's the point of supporting one specific use case while breaking others?

This feature can be implemented as a new flag for kata-runtime without breaking compatibility.
e.g.
$ cat /etc/docker/daemon.json
{
        "runtimes": {
                "kata": {
                        "path": "/opt/kata/bin/kata-runtime"
                },
                "kata-isolate-dev": {
                        "path": "/opt/kata/bin/kata-runtime",
                        "runtimeArgs": [
                                "--kata-device-mode=isolate"
                        ]
                }
}
$ docker run --privileged --runtime=kata-isolate-dev docker:dind ...

Kata is already interpreting the container spec quite liberally due to the extra abstraction layer involved. For instance, capabilities will not allow you to access the host system, but only the guest kernel in the VM. Why make an exception for devices?

@leoluk Kata does translate container spec to apply a different secure context. We handle the container spec as much as we can, and there are pieces we have to ignore because there is no way to handle them in a virtual machine context. Here you want devices that are translated now to be ignored. That is a huge difference.
docker run --privileged --runtime=kata-isolate-dev --device /dev/sda docker:dind
@AkihiroSuda The above is broken but I guess you just don't care?
All you guys want is to run docker in docker with kata, even in a half broken way. But I think it is better to provide a proper --privileged translation. How about we add kata's own privilege flag instead? When it is specified:

guest /dev, /proc and /sys are exposed to container in rw mode
container process has full linux capabilities
no selinux, no apparmor, no seccomp

This seems to match containerd's translation in a virtual machine context: https://github.com/containerd/containerd/blob/172fe90e55c3c3a452f9bec926d87ffda5ed01bf/oci/spec_opts.go#L1090-L1101
Then you can do:
$ cat /etc/docker/daemon.json
{
        "runtimes": {
                "kata": {
                        "path": "/opt/kata/bin/kata-runtime"
                },
                "kata-privileged": {
                        "path": "/opt/kata/bin/kata-runtime",
                        "runtimeArgs": [
                                "--privileged"
                        ]
                }
}
$ docker run --runtime=kata-privileged docker:dind ...

And docker run --runtime=kata-privileged --device /dev/sda busybox is still working as expected.
Then we can officially recommend kata-runtime --privileged over docker's --privileged flag. Users who want to use docker run --privileged should use kata-runtime --privileged instead.

docker run --privileged is still broken in many ways but we can recommend kata users to use kata-runtime --privileged instead. The main point is to have kata's own translation instead of working around containerd's translation that is broken for kata.

The issue with --privileged is that all containers will then be run as privileged when running in a pod.
In our k8s deployments we usually have multiple containers, and only the docker-dind container is started as privileged.  We just pass the docker socket to other containers to build/run/exec...

@darfux Could you explain a bit more on your use case? Are you running multiple containers in one pod, and only one of them is privileged?
We can define a container annotation to specify that a container in a pod is privileged. Does it work for your use case? Originally I thought a pod annotation or a new runtime handler is enough. But it seems in your use case we need a container level annotation instead.

I've edited my initial comment in the meantime, not sure you saw it.
But yeah, the use case is for CICD builds, with multiple containers per pod.
Usually we have a build container, specified by the customer running the job, and a we inject dind container if requested.

@darfux It seems a container annotation should be able to work for you, right? You don't use kata-runtime --privileged, but label the container spec with kata's privileged container annotation. Then kata knows that it should create the container as a privileged container in the guest.

While we might have a solution in kata, I still think cri/container is the right place to fix it. I've created https://github.com/containerd/cri/issues/1213 to ask for such a change there.

I'm not against an annotation, we already abuse them for other use cases.
However, we would have to modify the kubernetes pod spec, as annotations are at the pod level, not container. (As opposed to  privileged )

Sigh, that is too bad, not having container annotations. Then the containerd/cri change (containerd/cri#1213) seems more appealing than before. wdyt about it?

@bergwolf Sounds like a good approach to make the change in containerd/cri, as that's where the host devices are being listed and appended to the spec.
Only other concern - what about cri-o?

I'm not familiar with cri-o but I think it can make the same change. Right now the translation for privileged container is runc specific. It makes sense to have a different translation as runc is not the only runtime containerd/cri and cri-o support.

@dadux Does your usecase really need the dind container in the same pod?
I'm wondering you might be able to connect to dind in another pod (in its own VM) via headless service

@dadux Does your usecase really need the dind container in the same pod?
I'm wondering you might be able to connect to dind in another pod (in its own VM) via headless service

No because the dind in it's own VM still requires privileged, and in it's current state will have the host devices mounted in. If there was a vulnerability in dind or the kernel in the dind pod, then access to the host devices may occur.
It's also unnecessarily complicated.

@awprice My question was about the future status with some alternative to the current --privileged, not about the current status 😅
In other words - the alternative solution really needs to be container-level rather than pod-level?

We have a hard multi-tenancy requirements. pods (cicd builds) are isolated by namespace and have strict NetworkPolicies enforced, and when in Kata by different kernels.
Having the dind container in the same pod ( and vm ) makes sense, we only expose localhost.

@AkihiroSuda No, the solution would be pod level, as thats the lowest level of object that you can place an annotation on.
The logic in containerd/cri I imagine would work like this - If any of the pods have privileged AND there is a special annotation present to disable host devices, then don't append the host devices in the container spec.

Can we extend the CRI spec to add sys fs /dev/ /proc ... as rw ? AFAIK it's the only thing missing to emulate privileged without mounting all devices ?
The SecurityContext is per container too...

It looks like it's already been discussed, to redefine privileged in the CRI :
https://github.com/kubernetes/kubernetes/issues/44503
But nothing has happened. 😢

But yeah, the use case is for CICD builds, with multiple containers per pod.
Usually we have a build container, specified by the customer running the job, and a we inject dind container if requested.

Hi, @dadux , is it truly unacceptable that all containers in a pod are run as privileged?

@smarton6626 - yes.  In our case we're talking untrusted containers, with arbitrary code.
If the untrusted container is privileged, it  has capabilities to mount/attach devices, and access other containers in the pod's secret for instance.

@dadux - Is it avoidable? I mean, how about treating the whole pod as untrusted and not using secret in it?

The secrets was just an illustration, we treat the whole pod as insecure. But we also don't grant extra permissions to containers that don't need it. We drop all CAPS for those containers.
"privileged" mode just makes it a ton easier to break out of the container. Why would you want to allow that when you don't have to ?
There's good progress on the linked containerd/cri, with proposed solutions.

@dadux - Because I won't care they break out the container if I treat the whole pod as insecure. They are all in a VM, can do nothing to my host. Think about the VM instances in a public cloud (eg. EC2).

@smarton6626 - But we do care, having an extra kernel is awesome but not bulletproof.
https://www.cvedetails.com/vulnerability-list/vendor_id-7506/Qemu.html
Anyway, I don't think we're adding much value to initial problem here.

@dadux - Well, thanks for your input which provides me different perspectives. I'll think over that, although for now still believe VM is secure enough for isolating untrusted workloads.

privileged_without_host_devices PR for containerd/CRI (https://github.com/containerd/cri/pull/1225) recently got merged.
This will be available in containerd 1.3,
Proposal for porting over this to Moby/Docker: https://github.com/moby/moby/issues/39697

Is this solved?

Is this solved?

It's implemented in Kata and containerd, but has not yet propagated to Docker.

@haslersn, please, see http://lists.katacontainers.io/pipermail/kata-dev/2021-April/001819.html, there you'll find the explanation why the issue was closed.
Hi All,

# TL;DR

We're closing all open GitHub issues for Kata 1.x and 2.x that were created before 1st June 2020.

# Background

We have too many old open issues that are making it hard to identify the reallly important newer ones.

# Plan

Since a lot of those issues are now probably "stale", we intend to close all open GitHub issues opened before 1 June 2020.

# When?

This week some time.

# Why?

Closing these old issues will:

- Allow us to better manage the more recent issues

  We're in the process of categorising current issues with labels to allow us to manage the backlog more efficiently).

- Act as a forcing function for those who raised the issue

  The originator will get notified when the issue is closed, which will remind them of the issue and the problem. And if the issue is still important, they can re-open it with a single click.

# What is the impact of closing an issue?

Whether a GitHub issue is open or closed is simply a state field - you can open and close an issue as many times as you like: no information is going to be lost by changing the issue state to closed. By closing the old issues, we'll hopefully get the list of relevant open issues down to a more manageable number. And once labelled, will allow us to concentrate on fixing the issues rather than spending lots of time managing the issues themselves.

Of course, if an issue is closed, it's highly unlikely anyone will be working on resolving it, so...

# If one of your issues is closed

As mentioned above, if one of your issues is closed, please consider whether you are happy with this. If you think the issue should remain open, just click the "reopen" button and add a comment for us. It would be helpful if you could also mention whether the problem still exists with the latest 2.x version of Kata.

# Get involved

If you're interested in helping us manage the isue backlog, please get in contact [1], and maybe consider volunteering to help on the Kata rota [2].

Cheers,

James

[1] - https://github.com/kata-containers/community/blob/master/README.md#join-us
[2] - https://github.com/kata-containers/community/wiki/Review-Team-Rota

This one was rightfully closed, the feature in Kata is implemented and works!


Wrapping column settings by filetype

Hello,
is there a way to configure wrapping settings by filetype? If not, then this is a feature request.
Usecase: if I have both code and text files in my project, I would like to wrap text files but not code files.

@bpasero fyi

No, currently we do not support language specific settings, but we should.

#3533


Update Spanish translation

@malditogeek could you check this?

Perfecto! Gracias @TimothyGu :+1:

@malditogeek de nada :)
If anyone can review Chinese it would be cool to take a look at #5. Thanks!


Game crashes connecting to server trying to load cactus branches.

Use a paste site like Gist or Pastebin to dump any logs or crash reports.

Describe the bug
I crash connecting to my 1.16.5 forge server (version ) which has spawn next to a desert, so I just crash ever time I load in server. Updated both client and server from the Beta 8 Dynamic Trees to the Beta 9 and it fixed the issue.
To Reproduce
Load into a server with dynamic trees (version 1.16.5-0.10.0-Beta16) and dynamic trees plus (version 1.16.5-0.1.0-Beta8)
Rendering Glitches
I do have Optifine version 1.16.5_HD_U_G8.
Crashes

I am running the latest version of Dynamic Trees and Trees+.

Pastbin Link
https://pastebin.com/0mTJR1JY

Update to DT+ Beta 9.

they also literally said they updated to beta 9 and it fixed the issue, wtf

So your question is like asking “Forge for 1.16.4 does not load on 1.16.5, but when I update to Forge for 1.16.5 it works”.


flake: fix overlay not actually being applied

#3196 made some changes to flake.nix. In this line flake's waybar package was defined to be the one in nixpkgs because the function that was passed to genSystems argument was called with nixpkgs without the flake's overlay applied. I changed the genSystems function to pass nixpkgs that uses the flake's overlay so the package comes from the flake itself and not nixpkgs.

Thanks, apologies to @Arjun31415 and others.
This PR is crucial to merge (I was almost about to PR), because I made a mistake (again!) and this commit is borked.


Add HIV test results

FRONTEND PULL REQUEST
Related Issue

#6082

Changes Proposed

Test results shown on the test card will now be data driven so that all supported disease tests for the selected device are shown
Individual diseases like HIV or RSV will be filtered out if their feature flags are not enabled
Validation requirements for test results have been simplified so that the user can choose which diseases they report a result for or leave blank, as long as at least one result is selected
This means multiplex for COVID, Flu A, and Flu B

will no longer require the user to report both flu if only one of the flu results is reported
will allow the user to select mark individual results as inconclusive without having to mark all results as inconclusive

Additional Information

Verified with product that we don't actually need the complex validation requirements for individual disease combinations like multiplex devices
Will be following this up with another PR that adds HIV AOE

Testing

Verify that the test card still works as expected
Deployed on dev2

Screenshots / Demos
HIV only device

Covid, Flu A, Flu B multiplex

Covid only device

Did a first pass and left some questions! Good work on this

Updated to filter out RSV and HIV from the test result options if those feature flags are turned off. These screenshots are from my local deployment with the singleEntryRsvEnabled flag set to false
Multiplex device with RSV as a supported disease

Test card using that device but filtering out RSV

Things are generally looking good from a styling perspective, but I did notice some things (when comparing to the figma instance)

The order of the symptoms question vs. the pregnancy question is swapped compared to design (may not be incorrect, just calling it out)

The spacing between questions should be a consistent 24px (seems shorter between questions 3 and 4

The radio button labels for COVI-19 on the "Test card using that device but filtering out RSV" appear to be smaller than all other radio button labels

The padding between columns of radio button groups on multiplex card use cases should be greater for readability; 60px between columns is sufficient

@emyl3 I refactored the test queue to use a list element, but I can't reproduce the pregnancy bug so it might be best for us to fix that in a different ticket
https://github.com/CDCgov/prime-simplereport/assets/23287037/96606048-ce83-42e6-ad45-fd8bbd61b5ab
@changecourse regarding the order of pregnancy and symptoms, I used Kenny's Figma file here which has symptoms at the end because it expands with the additional questions if the user selected Yes.
I updated the spacing between the questions and between the columns of radio button groups. The radio button labels have the same height dimensions and font sizes

@mpbrown Sorry should have left steps to repro the pregnancy bug!
You need two tests in the queue.

one test leaves the pregnancy question blank
other test selects an answer
tabbing to the pregnancy question for the test that left it blank does not work

After doing some digging...
I think this is happening because the name for the radio group is not unique:
https://github.com/CDCgov/prime-simplereport/blob/ff837fa096f49fbe0baf5d504ec4629295d195bb/frontend/src/app/testQueue/TestCardForm/diseaseSpecificComponents/CovidAoEForm.tsx#L104
compare that with the has symptoms question that works:
https://github.com/CDCgov/prime-simplereport/blob/ff837fa096f49fbe0baf5d504ec4629295d195bb/frontend/src/app/testQueue/TestCardForm/diseaseSpecificComponents/CovidAoEForm.tsx#L114
Making it unique should fix the issue!


and choose the EntityFramework package - WHICH ONE!

and choose the EntityFramework package - WHICH ONE!  There are 10 to choose from.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b73b2e5a-9526-e664-a2da-556717d344e1
Version Independent ID: 93442b78-6869-139c-bdbc-9a1efa21b7a6
Content: Databinding with WPF - EF6
Content Source: entity-framework/ef6/fundamentals/databinding/wpf.md
Product: entity-framework
Technology: entity-framework-6
GitHub Login: @ajcvickers
Microsoft Alias: avickers

@nvnfdddddddd - Every participants in this repository are required to follow https://github.com/dotnet/EntityFramework.Docs/blob/main/CODE-OF-CONDUCT.md
I would suggest reviewing them.


[BUG] - SIM UICC configuration for Carrier Permissions does not work; alternative with GP

First, thanks for your great work!
Bug Report
Context

Device: sysmocom sysmoISIM-SJA2
Software: https://gitea.osmocom.org/sim-card/pysim.git, current master branch
OS: Ubuntu 22.04 LTS
Carrier Privileges (Y/N): n/a
Android Version: n/a
App Version: n/a
Git Hash: n/a

Description
When following main/docs/carrier-permissions.md, I got this error doesn't contain expected key command_store_ref_ar_do:
$ python pySim-shell.py -p 0 --csv card_Data.csv
Using reader PCSC[HID Global OMNIKEY 3x21 Smart Card Reader [OMNIKEY 3x21 Smart Card Reader] 00 00]
Waiting for card...
Info: Card is of type: UICC
Detected UICC Add-on "SIM"
Detected UICC Add-on "RUIM"
AIDs on card:
 USIM: a0000000871002ffffffff8907090000 (EF.DIR)
 ISIM: a0000000871004ffffffff8907090000 (EF.DIR)
 ADF.ISD: a000000003000000
 ARA-M: a00000015141434c00
Detected CardModel: SysmocomSJA2
Welcome to pySim-shell!
(C) 2021-2023 by Harald Welte, sysmocom - s.f.m.c. GmbH and contributors
Online manual available at https://downloads.osmocom.org/docs/pysim/master/html/shell.html
pySIM-shell (00:MF)> select ADF.ARA-M
null
pySIM-shell (00:MF/ADF.ARA-M)> aram_get_all
[
    {
        "response_all_ref_ar_do": null
    }
]
pySIM-shell (00:MF/ADF.ARA-M)> aram_store_ref_ar_do --aid FFFFFFFFFFFF --device-app-id E46872F28B350B7E1F140DE535C2A8D5804F0BE3 --android-permissions<PHONE_NUMBER>000001 --apdu-always
EXCEPTION of type 'ValueError' occurred with message: Dict [{'ref_ar_do': [{'ref_do': [{'aid_ref_do': 'FFFFFFFFFFFF'}, {'dev_app_id_ref_do': 'E46872F28B350B7E1F140DE535C2A8D5804F0BE3'}]}, {'ar_do': [{'apdu_ar_do': {'generic_access_rule': 'always'}}, {'perm_ar_do': {'permissions': '0000000000000001'}}]}]}] doesn't contain expected key command_store_ref_ar_do
To enable full traceback, run the following command: 'set debug true'
pySIM-shell (00:MF/ADF.ARA-M)> quit

Can you please check and expand the documentation when necessary?
How to Reproduce
Execute shell commands shown above.
Expected Behavior
Store the hash and access rule in ARA-M. As I used the same SIM card that is linked in the tutorial, I did not expect any error.
Actual Behavior
The error message shown above.
Screenshots
n/a
Additional Context
n/a
Possible Fix
Workaround: Using gp worked for me.
I used the version from https://github.com/herlesupreeth/CoIMS_Wiki with the following command:
java -jar gp.jar --key-enc KIC1 --key-mac KID1 --key-dek KIK1 -a 00A4040009A00000015141434C0000 -a 80E2900033F031E22FE11E4F06FFFFFFFFFFFFC114E849B63A7B96E571F788FC5845C4AA3C520D18E8E30DD00101DB080000000000000001

It adds the rule for the hash E849B63A7B96E571F788FC5845C4AA3C520D18E8 (which is the one from your tutorial).
However, this utilizes raw APDUs, which is not very convenient.

While I'm not sure why this error happend, i guess it depends on the pysim version as it works with my test installation we good a nice pull request that updates the documentation to GP way to set the signature.
https://github.com/omnt/OpenMobileNetworkToolkit/blob/main/docs/carrier-permissions.md
@buchbergerd does this solve the issue for you?

@buchbergerd what you describe sounds like a pySim-shell bug.  Did you report it upstream? Are you sure it occurs with current pysim.git master?


Menu dragging with content

For example I have left REFrostedViewController.
With standard parameters.
If I drag menu to the right - it resize yourself but not content.
If it possible to disable this gestures without disable anything else or enable resize content? I prefer second way :)

Hi, and thanks for this awesome content. :)
I wanted to resize the dragMenu according to the screen orientation, I tried
CGFloat screenHeight =[UIScreen mainScreen].bounds.size.height/4;
CGFloat screenWidth =[UIScreen mainScreen].bounds.size.width;
self.limitMenuViewSize=YES;
[self resizeMenuViewControllerToSize:CGSizeMake(screenWidth, screenHeight)];
in didViewRotate method in rootview controller. but It's not happening. Is there anything I am missing here?


RMANTREE Config issues.

Configuration of the addon is unnecessarily difficult.  Setting an env variable for RMANTREE is too hard for most users and where 99% of the config issues come in.  Here's what I'm proposing.

Have an "RMANTREE" preference in the addon preferences.  This is where RMANTREE comes from.
If that is not set try to pull from env variables or latest found as we already do.
Users can set the RMANTREE setting the preferences manually.  Maybe put a button in there to "pull from latest installed"...

Brian,
I think that would be a great idea.

Here's what I came up with:  Basically a two part preference.  The default is still just "Grab the latest installed to default location"
First the user chooses how to get renderman

Choose from what's installed (default).  This will give them a dropdown of RenderMan versions that installed to default location (/Applications/Pixar on osx).  Or by default get the newest installed version.
Get from RMANTREE environment variable for power users who use env variables (also gives feedback what that is set to)  In hindsight not many blender users use env variables but still want to include it.
Set install location manually with file browser

Done in devel branch.

I don't have these preferences. Is it normal ?

You can't run just the preferences module as it's missing all the commands that initialize the plugin. Is here any particular reason you're trying to run it by itself?

Ok. I was trying to run it by itself because I wanted to understand why I don't have the preferences.

Ok I understand better now. The issue is that I get this error when trying to enable the add-on :
Traceback (most recent call last):
  File "/home/caetano/Installed_programs/blender/2.78/scripts/modules/addon_utils.py", line 349, in enable
    mod.register()
  File "/home/caetano/.config/blender/2.78/scripts/addons/RenderManForBlender-master/__init__.py", line 121, in register
    load_addon()
  File "/home/caetano/.config/blender/2.78/scripts/addons/RenderManForBlender-master/__init__.py", line 95, in load_addon
    if guess_rmantree():
  File "/home/caetano/.config/blender/2.78/scripts/addons/RenderManForBlender-master/util.py", line 553, in guess_rmantree
    "Error loading addon.  RMANTREE %s is not valid.  Correct RMANTREE setting in addon preferences." % rmantree)
  File "/home/caetano/.config/blender/2.78/scripts/addons/RenderManForBlender-master/util.py", line 59, in throw_error
    raise ImportError(msg)
ImportError: Error loading addon.  RMANTREE  is not valid.  Correct RMANTREE setting in addon preferences.

But The preferences don't appear until the add-on is enabled, so how am I supposed to "Correct RMANTREE setting in addon preferences" ?

I tried
export PATH="${PATH}:/opt/pixar/RenderMan-Installer-ncr-21.4/bin"
export RMANTREE="${RMANTREE}:/opt/pixar/RenderMan-Installer-ncr-21.4"

but it didn't seem to change anything. I still can't enable the add-on. Any idea ?


namegen

An API script to allow users to randomly generate names based on gender and nationality. In the future, additional nationalities will be added as will the ability to generate a name completely at random (no gender/nationality input required)

Going to submit pull request for updated (and much better) version - so closing this pull request


Update un-versioned dependencies

Motivation
Updates dependencies not covered by dependabot
Changes

Update dependencies github.com/chaos-mesh/chaos-mesh/api, golang.org/x/exp and google.golang.org/genproto/googleapis/rpc

Test Plan
existing tests pass
TODO

[x] Explain motivation or link existing issue(s)
[x] Test changes and document test plan
[x] Update documentation as needed
[x] Update changelog as needed

bors try

bors merge

bors merge


¿Can I upload a file in memory to Azure File?

Which service(blob, file, queue) does this issue concern?
File
Which version of the SDK was used? Please provide the output of pip freeze.
azure-storage-file==2.1.0
What problem was encountered?
I have created a .zip file in memory. If I write it into a file i can dowload it but I can´t find any way to upload it from the 'in memory' file
Have you found a mitigation/solution?
Note: for table service, please post the issue here instead: https://github.com/Azure/azure-cosmosdb-python.

Hi @pobs93
You can call create_file_from_path() API.
def run_sample():
    try:
        # Create the FileService that is used to call the File service for the storage account
        file_service_client = FileService(
            account_name='emilystageaccount', account_key='youraccountkey')

        # Create a share called 'quickstartshare'.
        share_name = '''quickstartshare'''
        file_service_client.create_share(share_name)

        # Create Sample folder if it not exists, and create a file in folder Sample to test the upload and download.
        local_path = os.path.expanduser("~/Documents/Sample")
        local_file_name = "release.zip"
        full_path_to_file = os.path.join(local_path, local_file_name)

        print("Temp file = " + full_path_to_file)
        print("\nUploading to File storage as file" + local_file_name)

        # Upload the created file, use local_file_name for the blob name
        file_service_client.create_file_from_path(
            share_name, None, local_file_name, full_path_to_file)

        # Download the file(s).
        # Add '_DOWNLOADED' as prefix to '.zip' so you can see both files in Documents.
        full_path_to_file2 = os.path.join(local_path, str.replace(
            local_file_name,'.zip', '_DOWNLOADED.zip'))
        print("\nDownloading file to " + full_path_to_file2)
        file_service_client.get_file_to_path(
            share_name, None, local_file_name, full_path_to_file2)

        sys.stdout.write("Sample finished running. When you hit <any key>, the sample will be deleted and the sample "
                         "application will exit.")
        sys.stdout.flush()
        input()

        # Clean up resources. This includes the container and the temp files
        file_service_client.delete_share(share_name)
        os.remove(full_path_to_file)
        os.remove(full_path_to_file2)
    except Exception as e:
        print(e)

# Main method.
if __name__ == '__main__':
    run_sample()


[Proposal] Query function need to add

[ ] Query multiple sign account's owner public key and weight information

[ ] Query who approved the proposal after released

[ ] Query the account's proposal list(not approve/approved/overtime)

Not implement on node side, will implement on custom app.


Change SliceInfo::new() to return a Result

This is a last-minute change for 0.11.0. I think it would be better for SliceInfo::new() to return Err instead of panicking if D is not consistent with indices. It's more flexible for the caller and is more similar to .into_dimensionality().
This change doesn't affect #391 and should merge cleanly.
Edit: I initially chose to make SliceInfo::new() panic becuase I thought it would be weird for a method named new() to return a Result instead of the object itself. However, there is precedent for returning a Result from new(), e.g. regex::Regex::new().

Thanks!


Add 65k GKE benchmark

This PR adds a benchmark for GKE on 65,000 nodes scale
This is following up on the recent GKE announcement for support clusters with 65,000-nodes scale.
It benchmarks GKE on a (simulated) AI workload using using Terraform and ClusterLoader2.
I didn't include it as part of benchmarks directory as the benchmarking process here is different (using CL2) and has a different infra setup.

Agree with @achandrasekar that it would be good to put this under the existing benchmarks/ directory. Is it possible to make that change?

Agree with Ashok Chandrasekar that it would be good to put this under the existing benchmarks/ directory. Is it possible to make that change?

I didn't include it as part of benchmarks directory as the benchmarking process here is different (using CL2) and has a different infra setup.

I think tooling being different is not a blocker, but would be good to consoildiate eventually

I had an attempt at reorganizing the benchmark, here is a summary of the changes:

put existing infra under infra/gpu-cluster
moved infra part under infra, put under folder 65k-cpu-cluster
moved CL2 part under benchmark/tools
renamed current README to gpu-benchmark.md
moved 65k README and renamed into 65k-nodes-simulated-ai-workload
added a root level README for benchmarks/ directory

Thanks for refactoring @besher-massri! @annapendleton can you take a look to make sure this doesn't affect the existing benchmarks?

Thanks for the review and comments! I pushed another commit, changing from gpu to accelerator and highlighting that both GPU and TPU clusters are supported. Changed some naming, minor rewording as well. Let me know if it's okay.

That looks great, thanks! Please make sure to resolve anything lingering with Andrew prior to submission :)

/gcbrun


add filter monitoring by budgetId and staff name

Changes Unknown when pulling 3a8e1f68f0112a138a111dca780816bb740c5c87 on indriHutabalian:dev into ** on danliris:dev**.


Retry move of directory when updating agent version

It is possible when doing an auto-update of an agent that the file handle lock on the bin/ directory will take remain some time after the process is shutdown, especially on Windows hosts. This can cause the update process to fail as it tries to move the current directories out of the way and install the new versions.
We can reduce the likelihood of a failed update by attempting retires on the move operation.

Perhaps a better solution would be to have the run.sh script use the new version of the bin folder instead of moving it.
So for example:
Agent starts with bin/ and externals/
During the update, instead of moving files around, we download the new binaries to bin.(VERSION) (similar to what we do now).
Then, the run.sh script knows to look for the highest version available and just runs the .dll from that folder.

I think that would work if we could guarantee no customer is relying on accessing the binaries through bin/ or externals/. I am not sure we can do that. There are a few places where we reference git or node through $AGENT_ROOT/externals. The run.sh script is not the only entry point to the agent.

Another option as I think about it might be to use rename instead of move. I am not sure if rename works better with file locks.

Hi @jtpetty and @alex-peck
I quite like the idea of downloading to new directories - but as you say there are potential impacts on other customers who make direct references to those files.
Some further thoughts:

Check for any residual file handles. Log the existence of such file handles. Close any residual file handles. Perhaps handle is useful here but I don't know if this is installed on windows systems by defaults and especially not sure when it comes to docker windows-servercore and nanoserver. So this is probably a non-runner.

Poll for any residual locks on that folder and simply wait for them to be closed by the os. Although, hopefully such locks are closed.

@jtpetty This is related to our current agent work stream.


Backport of PR #124

Backports PR #126
Added continuous query to RawPointerMap.

Test PASSed.

Test PASSed.


New 33Across Analytics Adapter 1.0.0

🏷 Type of documentation

[x] new Analytics adapter
[ ] update bid adapter
[ ] new feature
[ ] text edit only (wording, typos)
[ ] bugfix (code examples)
[ ] new examples

📋 Checklist

[x] Related pull request https://github.com/prebid/Prebid.js/pull/10635
[ ] For new adapters check submitting your adapter docs

Thanks, @muuki88 - I've fixed the linting errors.

this is in 8.24


Update Terraform Version

Terraform Version & Prov:
AFT Version:
1.7.0
Terraform Version & Provider Versions
Please provide the outputs of terraform version and terraform providers from within your AFT environment
terraform version
v0.15.5

terraform providers
+ create

Bug Description
I would like to update the terraform version being used by CodeBuild to the newest version. The version it is using now is the version v0.15.5. How do I do it? Do I need to change anything directly to AWS, update the value of the variable /aft/config/terraform/version do the trick, or there is a way to do it via .tf?

You have a variable in the aft module which allow to change it.

If I change it there it will start to use this version instead? No need for more settings?

Yep but better to take a look by yourself, https://github.com/search?q=repo%3Aaws-ia%2Fterraform-aws-control_tower_account_factory+terraform_version&type=code

@raben0 thank you for reaching out.
You can specify the Terraform version you want to use in your environment by setting the variable terraform_version during AFT deployment / AFT update.


CF Release

Updates
Changed

cf-expandables: Fixed bottom spacing of the expandable content

Review

@cfpb/front-end-team-admin

I am a bot
After this PR is merged, I will instruct Travis to automagically perform the following steps:

Increment Capital Framework's version per our guidelines.
Add a timestamped entry to the changelog with the new version and its changes.
Tag the release and push it to GitHub.
Publish both capital-framework and any individually updated components to npm.
Update both canary, our development branch, and master, our release branch.

If I do something wrong, blame a human.

:+1:


[FEATURE]Can customize third-party openai mirror websites

Want to be able to support custom third-party openai sites, for example
https://api.openai-sb.com/
https://api.openai-proxy.com/
https://openai.api2d.net/

you can add the URL to the basePath:


Add Navigation-pop

when using the onFile function, clicking the check icon did not close the camera. I believe that this would be a desired behavior.
I also centralized the return icon

Dear i'll try ti add return icon.. can share how you do that i'll try to add in setState { navigator.pop } but dosen't have any effect ..

fix new version 2.0.0


Java 11.0.1

Hey Luck, testing Java 11 and it seems this plugin is the only one shooting an error for me.
Can't even see it in the logs, seems to be a client side thing on the local machine:
https://i.imgur.com/Rf461Qz.png
This is me loading up the server on my test box, all other plugins work fine (over 40), just luck perms being weird for some reason.
Previously was using Java 8

Pretty much an exact duplicate of #1290.


Various small things

At start-up, doing SpeciesInit() leads to:

#UserWarning: no type annotations present -- not typechecking species.util.dust_util.check_dust_database

Would it be good to convert silently resolution in database.add_object() to a float? Currently:

type of argument "spec_res" must be one of (float, NoneType); got numpy.int64 instead

Keeping resolution an integer makes it easy to combine it with filenames, for instance, and one often thinks of integer resolution.

The documentation says about add_object(): Function for adding the photometry and/or spectra of an object to the database and the tutorial writes: "The use of add_object is incremental so when rerunning the method it will add new data in case a filter or spectrum name hadn’t been used before, or it will overwrite existing data in case a filter or spectrum name does already exist" but actually it replaces the data when adding spectral data in two different calls:

database.add_object(object_name = 'myPMO', parallax=(100,0.1), spectrum = {'onepart': ('file1.dat', None, 1000)})
database.add_object(object_name = 'myPMO', parallax=(100,0.1), spectrum = {'anotherpart': ('file2.dat', None, 1000)})

After the second call, database.get_object('myPMO').open_box() will only list the 'anotherpart' spectral data. If the parallax is not set in the second call, it does keep it from the first (as should be).
Honestly, I have found it up to now practical to have the data overwritten and then in FitModel not have to worry about (because I forgot to…) including only the datasets I want, but at other times it can lead to unpleasant surprises. Maybe provide an overwritedata= (by default False) keyword?

Two tiny things in species/util/box_util.py:
" but the argument of 'model' is set to "
→
"but the argument of 'model' is set to "
:nerd_face:

"is differently implemented with"
→
"is implemented differently with"

When calling database.list_content(), it would be nice to print the stored resolution of the spectral data.

Similarly, it would be useful to output the (lowest, average, highest) resolution in the model when doing modelbox_full.open_box()

In ./species/plot/plot_mcmc.py:

print(f"Plotting the posterior: {output}...", end="", flush=True)
→
print(f"Plotting the posterior: {output} ...", end="", flush=True)

This would make copying the filename from the terminal easier: without a space, the three dots get selected when double clicking on the name :D.

In corner plot, for the column title at least for ATMO:
ad_index = {}
→
$\gamma = {}$

Put the (e.g.) "1e-16" as "10^{-16}" into the parenthesis with the units of flux density (classical way) instead of on top of the plot (defensible but less elegant and takes up one whole line for a number that is not of detailed interest)

When in plot_spectrum units gets implemented, or until then as an option, to have erg/cm²/s/Å units for the flux density (could be called "cgs-like" or "Swedish-style" (svensk=True?) because of Å :wink:)

Thanks!

Hi Gabriel! I think this has all been implemented in the meanwhile? For 10, you can use the new units parameter in plot_spectrum. For 9, this will get adjusted I think when setting the ylim parameter. I have added the spec_res attribute to the ModelBox in the latest commit f1f6807fc78669f7b9bc4d9a7e184a35c8a83a69. I hope this helps!

Hello Tomas! Very nice—thanks a lot!

I will try out 8. but if you say it is done, of course I believe you.
Excellent for 9.! Indeed, it is necessary and sufficient to set ylim. I tried other options and it indeed all seems to work :heavy_check_mark:.
Thanks for spec_res. This will get saved in the database automatically I guess, yes? Will you distinguish somehow between the resolution and the sampling…?
For 10.: Thank you. Maybe a missing popular flux density unit is erg/s/cm²/Å (only …/Hz is available, which is not a trivial transformation). Also, the linguist and culture-lover in me would suggest using "AA" instead of "A" ( :norway: :denmark: :sweden:), and of offering the synonyms "Å" and "µm" :nerd_face:… Sorry but "Å" is really not pronounced the same as "A" :smile:!

Thanks :grin:! Sorry, last tweaks ("sigh once, give lots of users even more comfort"):

When specifying e.g. units=("AA","W m-2 µm-1"), the AA does not get converted and the -2 and -1 are not exponents in the plot.
The µ, Å, AA are not "advertised" in the docstring of species.util.data_util.convert_units. Maybe just addd a note at the end of the entry for units_in (and use "AA" by default in the documentation because it is shorter to type…)
Offer the synonyms "erg/s/cm^2/AA", etc. (with slashes) for typing convenience? It is a bit difficult to suppress the "LaTeX" reflexes and use spaces and no ^… :nerd_face:. But this is really a small one. Fortunately, one does not change the units too often :).

Also, sorry but when using units=("µm","erg s-1 cm-2 angstrom-1"), there must be a factor of cm/µm=1e-4 dividing instead of multiplying in the flux density because I plot some data with "W m-2 µm-1" and get F_lambda = 1e-16 W/m²/µm but supposedly F_lambda = 1e-9 erg/s/cm²/Å; the difference is 1e-8 = (1e-4)² :smile:… a classic.

Excellent! Thank you very much. While marking this as closed, I would just suggest in the species.util.data_util.convert_units docstring:
…
"Jy", "MJy"). One can use "um" or "µm" interchangeably, and similarly "AA", "Å", "A", or "angstrom".


failed to set thread name with 'io.flutter.raster'

Steps to Reproduce

Run flutter create hello.
cd hello
Run flutter build linux --debug
cd build/linux/debug/bundle
run ./hello with gdb
ctrl+c and run info thread to show threads name

Expected results:
  7    Thread 0x7fffe2f3a700 (LWP 518213) "io.flutter.raster"         0x00007ffff5df643e in epoll_wait (epfd=8, events=0x7fffe2f39e58, maxevents=1, timeout=-1)
    at ../sysdeps/unix/sysv/linux/epoll_wait.c:30

Actual results:
  6    Thread 0x7ffff0d3b700 (LWP 518212) "io.flutter.ui" 0x00007ffff5df643e in epoll_wait (epfd=6, events=0x7ffff0d3ae58, maxevents=1, timeout=-1)
    at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
  7    Thread 0x7fffe2f3a700 (LWP 518213) "hello"         0x00007ffff5df643e in epoll_wait (epfd=8, events=0x7fffe2f39e58, maxevents=1, timeout=-1)
    at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
  8    Thread 0x7fffe2739700 (LWP 518214) "io.flutter.io" 0x00007ffff5df643e in epoll_wait (epfd=10, events=0x7fffe2738e58, maxevents=1, timeout=-1)
    at ../sysdeps/unix/sysv/linux/epoll_wait.c:30

pthread_setname_np with 'io.flutter.raster' is not working on linux.
int pthread_setname_np(pthread_t thread, const char *name);
    The thread name is a meaningful C language string, whose length is restricted to  16  characters

It seems that we need to update with a new name for raster thread on linux.

Hi @lihui7115
I do not find any reference in the docs
flutter desktop and desktop shells
about the use of gdb as debugging tool
or the expected output of these debug, threads specifically.
Could you please reference the source for these informations
Thank you

Hi @iapicca
I think there is no information about this issue from these docs you provided. Actually, I just deep into the flutter engine source code, and find this small limitation about thread name on Linux.
https://github.com/flutter/engine/blob/ede658e2d1e976152370f404b6cff874f42a6611/shell/common/thread_host.cc#L23
Maybe I should not post this issue here(flutter framework repository), but there is no 'issues' section on flutter engine repository.

Hi @iapicca
I think there is no information about this issue from these docs you provided. Actually, I just deep into the flutter engine source code, and find this small limitation about thread name on Linux.
https://github.com/flutter/engine/blob/ede658e2d1e976152370f404b6cff874f42a6611/shell/common/thread_host.cc#L23
Maybe I should not post this issue here(flutter framework repository), but there is no 'issues' section on flutter engine repository.

Not familiar with gdb and threads, however, I got the  following results  when trying to reproduce
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/taha/AndroidStudioProjects/hello/build/linux/debug/bundle/hello hello
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=2, argv=0x7fffffffdd28) at /home/taha/AndroidStudioProjects/hello/linux/main.cc:4
4	  g_autoptr(MyApplication) app = my_application_new();

flutter doctor -v
[✓] Flutter (Channel master, 1.25.0-5.0.pre.108, on Linux, locale en_US.UTF-8)
    • Flutter version 1.25.0-5.0.pre.108 at /home/taha/Code/flutter_master
    • Framework revision 995aa138cd (11 hours ago), 2020-12-08 09:18:02 +0530
    • Engine revision b0d31ae166
    • Dart version 2.12.0 (build 2.12.0-125.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /home/taha/Code/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /home/taha/Code/sdk
    • Java binary at: /home/taha/Code/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Linux toolchain - develop for Linux desktop
    • Ubuntu clang version 11.0.0-2
    • cmake version 3.16.3
    • ninja version 1.10.0
    • pkg-config version 0.29.2

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).

[✓] VS Code (version 1.51.1)
    • VS Code at /usr/share/code
    • Flutter extension version 3.16.0

[✓] Connected device (2 available)
    • Linux (desktop) • linux  • linux-x64      • Linux
    • Chrome (web)    • chrome • web-javascript • Google Chrome 87.0.4280.88

! Doctor found issues in 1 category.


Compatability with BeatLeader

Would be nice to have this tool compatible with BeatLeader as well so it's easy to download lots of maps at once for ranked practice and etc. :3

Still looking into this. I don't think it's going to be easy at all to integrate this into beat leader because of how the links are generated on the page to downloads.


Generate Financial Reporting - Bug Fixes

Client reported that financial reporting flag wasn't working properly. This has been fixed and tested to ensure that it's working as intended.
Client asked that there be an option added for ALL BOOKING EVENTS to the generate modal, and that there be an ALL EXAMS option as well. Not merging both options. This has been remedied and the UI to the modal has been updated to have a dropdown with filter options in the dropdown.
Client reported that the ITA and Non-ITA filters were reporting appointments and not just exams. This has been fixed and tested.

Forgot to add to the description, I fixed a small bug with creating rooms in the admin panel, hence the file being included here.


Clarify sampler array indexing

Thanks Gregg for putting together WebGL2Fundamentals. It's a great overview.
Unfortunately there's one small but significant bug. In WebGL 1.0 it was optional whether indexing of sampler arrays with variables was supported. Shaders using this feature might compile, or might not, depending on the capabilities of the underlying hardware.
The GLSL ES 3.00 specification forbids indexing of sampler arrays with anything except a constant. See sections 12.29 "Samplers" and 12.30 "Dynamic indexing".
We're looking into adding a WebGL 2.0 extension adding this functionality back in a guaranteed manner.

Thanks Ken.
I removed that part here 857b25168b676cc885e8dfe29131a8f6a4ed0db9

Thanks Gregg!


Add calculated field recorded_end_date to AudioRecording's filter options

Request: and calculated field recorded_end_date to the filter options in the AudioRecording model.
This field would be defined as:
recorded_end_date = recorded_date + duration_seconds

This field will be useful in completing the saved search story. E.g. find all recordings that have content past a certain date.

:+1: - more complicated than I expected but very cool. Cheers.

:smiley: yeah, I had to use arel to construct the query.
It's like there's a gremlin in the system that sees your issues, and fixes them.....

Be aware that that dates you send must be in utc, and prefer using Z over +0000. I had trouble getting some offsets and time zone names to parse correctly, since the filter system doesn't use ActiveRecord's datetime conversion.

hmm... that could represent a serious problem. Does this affect all date fields used in all filters? Or just this custom one?

I think this would affect all date fields in filters (other parts of API should be fine).

One last question: does it throw an error with bad timezones it fails to parse or does it silently continue?

I hate to say it, but I think it silently continues with whatever it thought the datetime is :disappointed:

This... probably... definitely needs a bit more work...

It is a separate issue, tracked in #263


Support templates for branch names

When using the issue viewer to create/checkout a branch, I'd like to be able to adjust the template that's used. Currently, it's
{github-user}/issue{issue-number}

I want to be able to customize that.

It seems like maybe there's code to support that, but I don't know enough about VS Code extensions to really understand it.
https://github.com/microsoft/vscode-pull-request-github/blob/main/src/issues/currentIssue.ts#L158-L163

Sweet! I looked for that, but didn't find it, for some reason. 👍


Fixed unwanted execution of dragStart event handler inside dndDraggable directive

When you use both directives (dnd-draggable and dnd-nodrag) on the same element, dnd-nodrag will not work as expected because it will only prevent the dragstart and dragend events from bubbling up but won't prevent the rest of element event handlers from firing up.

Thanks for your PR! This is an elegant solution I did not know about 👍


Unable to connect

Hello, I have tried to use pycai's (not AIOCai) connect() on another computer with using my Guilded bot. though on another computer, I get the error message here

(Note: the output is reversed)
I'm using characterai==1.0.1 and websockets==13.1
This machine is running Debian testing.
The previous machine used websockets==12.0 and characterai==1.0.0 and runs Debian 12 stable.
I have downgraded the modules and still have the same issue.
Code:
from characterai import pycai
client = guilded.Client()
#C.AI
caitoken = os.environ["CAItoken"]
caibot= os.environ["CAImodel"]
while True:
    try:
        cai = pycai.Client(caitoken)
        me = cai.get_me()
        chat = cai.connect()
        chats = []
        break
    except:
        print(traceback.format_exc())

If I go ahead and remove the init parameter, it throws a 403 error from the web socket.


Split base16-builder cli from core

Hello,
What would you think to split base16-builder in two projects. One for the core and one for the CLI. If someone, someday, want to create a GUI version or another tool. He could use the core part of base16-builer. In C#, i would ask for switching for one project to a group of projects (a PCL, a CLI, why not a wpf, etc).
Basically, can we add more modularity in base16-builder?

base16-builder already modular. It isn't documented, but it's possible to install base16-builder locally:
npm install base16-builder

... And then reference buildTheme:
import { buildTheme } from 'base16-builder'

buildTheme(...)

Ok. I learnt something.

Closed by 5aa25ae3a5c573cce1943caa1da6f18a42ce01c6. It's not ideal to reference a ticket as documentation but it's good enough for now.


CMake GLPK build is broken

Hi, looks like building with GLPK support using CMake is broken. Getting errors about undefined symbols for SetupGlpkEnvAutomaticDeletion.
What version of OR-Tools and what language are you using?
Version: v9.3
Language: Python 3.9
Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)
pywraplp.Solver.CreateSolver("GLPK")
What operating system (Linux, Windows, ...) and version?
Debian Bullseye
What did you do?
$ apt-get install --no-install-recommends -y \
      git ninja-build pkg-config build-essential autoconf libtool lsb-release ccache

# build or-tools
$ cmake -S. -B build -G "Ninja" -DCMAKE_BUILD_TYPE=Release \
    -DBUILD_DEPS=ON -DUSE_GLPK=ON -DBUILD_GLPK=ON -DBUILD_FLATZINC=OFF \
    -DBUILD_PYTHON=ON -DBUILD_SAMPLES=OFF -DBUILD_EXAMPLES=OFF \
    -DBUILD_TESTING=OFF -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
  && cmake --build build --parallel 8 -v

# smoke test
$ pip install build/python/dist/*.whl \
  && python -c "from ortools.linear_solver import pywraplp as p; p.Solver.CreateSolver('SCIP')" \
  && python -c "from ortools.linear_solver import pywraplp as p; p.Solver.CreateSolver('GLPK')"

What did you expect to see
Nothing :)
What did you see instead?
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/src/or-tools/build/python/ortools/linear_solver/pywraplp.py", line 13, in <module>
from . import _pywraplp
ImportError: /src/or-tools/build/python/ortools/linear_solver/../../ortools/.libs/libortools.so.9: undefined symbol: _ZN19operations_research29SetupGlpkEnvAutomaticDeletionEv

Anything else we should know about your project / environment
I was able to come up with a patch that fixes the undefined symbols. If it looks reasonable I can submit a PR.
diff --git a/cmake/cpp.cmake b/cmake/cpp.cmake
index 414d878242..dbef70ecad 100644
--- a/cmake/cpp.cmake
+++ b/cmake/cpp.cmake
@@ -27,6 +27,7 @@ if(USE_COINOR)
 endif()
 if(USE_GLPK)
   list(APPEND OR_TOOLS_COMPILE_DEFINITIONS "USE_GLPK")
+  set(GLPK_DIR glpk)
 endif()
 if(USE_PDLP)
   list(APPEND OR_TOOLS_COMPILE_DEFINITIONS "USE_PDLP")
@@ -226,6 +227,7 @@ foreach(SUBPROJECT IN ITEMS
  constraint_solver
  ${PDLP_DIR}
  ${GSCIP_DIR}
+ ${GLPK_DIR}
  glop
  graph
  gurobi
diff --git a/ortools/glpk/CMakeLists.txt b/ortools/glpk/CMakeLists.txt
new file mode 100644
index<PHONE_NUMBER>..25e25f38f4
--- /dev/null
+++ b/ortools/glpk/CMakeLists.txt
@@ -0,0 +1,24 @@
+if(NOT USE_GLPK)
+  return()
+endif()
+
+file(GLOB _SRCS "*.h" "*.cc")
+
+set(NAME ${PROJECT_NAME}_glpk)
+
+# Will be merge in libortools.so
+#add_library(${NAME} STATIC ${_SRCS})
+add_library(${NAME} OBJECT ${_SRCS})
+set_target_properties(${NAME} PROPERTIES
+  CXX_STANDARD 17
+  CXX_STANDARD_REQUIRED ON
+  CXX_EXTENSIONS OFF
+  POSITION_INDEPENDENT_CODE ON
+  )
+target_include_directories(${NAME} PUBLIC
+  $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}>
+  $<BUILD_INTERFACE:${PROJECT_BINARY_DIR}>)
+target_link_libraries(${NAME} PRIVATE
+  absl::strings
+  $<$<BOOL:${USE_GLPK}>:GLPK::GLPK>)
+#add_library(${PROJECT_NAME}::glpk ALIAS ${NAME})

ok got it
https://github.com/google/or-tools/commit/42e0f3a2c1a277234a774c560172d98e940c13d9 enable it for the Bazel based build but nothing was done for the CMake based build -> break of the build.


perf(index): optimize performance

Reduce excess memory overhead

Coverage remained the same at 100.0% when pulling 8f8d22dc25accbc9e46b47838cad8bf1a76f0ae5 on BlackHole1:patch-1 into bbb22109f11888910af1965c903aa6b1712c3922 on johnwquarles:master.

Thank you for this contribution!
I think this is a welcome addition to the code. One thought I have about the second use of Object.create(null) is that it is being used to create object that is ultimately returned as the keyMirror object.
I think that consumers of this package may expect to be able to use Object prototype methods upon the object that is returned. So while the optimization is great for keysSeen, I think it will be best to keep the object literal as the third argument to the reduce call.

yep, I didn't think about this. Thank you very much for your answer


[not solved, WIP] x-editable not working with bootstrap 3.0 final

Hello,
I saw a discussion here (https://github.com/vitalets/x-editable/issues/318) about this but I could not find any solution.
x-editable is not working with bootstrap 3.0; Chrome gives an error:
Uncaught TypeError: Cannot use 'in' operator to search for 'value' in undefined
This at bootstrap-editable.js:981
Do you have any solution for this?
Thanks a lot,
Xavier

Beware of using the stock CDN version of Bootstrap3 (i.e. the 'Download Bootstrap' version from their front page - as it does /not/ contain the tooltip and popover extensions - both required for bootstrap-xeditable to load correctly.
Instead, generate a customized version of Bootstrap, ensuring you add these missing dependencies.

why in my project doesn't work?
VM1815:4 Uncaught TypeError: $(...).datetimepicker is not a function

-->previous thread: guys, it seems you are including bootstrap-editable.js before bootstrap.js file.
It will not work as xeditable js depends on bootstrap.
You should include bootstrap-editable.js after bootstrap.js.
Please re-check and let me know..
@vitalets already tried this one, popup mode on my combodate is still not working

@markymar25 Check the jquery includes as well. I was in the same situation and my problem was that there was a second include of  jquery.js after bootstrap.js

I am still getting some error. I am trying to implement inplaceeditor in an modal.

hai,Have you solve your problem? i have got

I am still getting some error. I am trying to implement inplaceeditor in an modal.

This is the script that I am using for bootstrap.
have you solve your problem?i met this same problem with you


Ingress tightening

Centralize ingresses and firm up security checks. Rate-limiting for sure, and possibly also client-IP matching.

Closing in lieu of https://github.com/nre-learning/antidote/issues/43


training cnn network

i have built a cnn network which im trying to train with my own data. but when i run the code i get the next error:
File "C:/linear regression/new_net.py", line 92, in <module>

ValueError: Cannot feed value of shape (500, 99999) for Tensor 'X:0', which has shape '(500, 4392)'

and i dont know what value i need to fix. my data is built of 100,000 rows on 4392 column. i reshape each row to 61*72 pixels which means i have 100,000 images. i take 80,000 to the training and 20,000 to the test. and using batch size of 500. i dont know what is wrong with the code if someone can help please!!!
here is my code:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import csv
import time
import os
import urllib

import numpy as np
import tensorflow as tf
#input images- chek if i need the transpose command
data_in = np.transpose(np.loadtxt(open("images.csv"), delimiter=","))

images_training="images.csv"

train_X=data_in[0:80000,:]
test_X=data_in[80001:100000,:]

test_images=test_X[:,1]
test_labels=test_X[:,0]

n = train_X.shape[1]
l = test_X.shape[0]

n_epochs = 10
batch_size = 500
regions=50
height = 61
width = 72
channels = 1

n_inputs = height * width #4392

X = tf.placeholder(tf.float32, shape=[batch_size, n_inputs], name="X")
X_reshaped = tf.reshape(X, shape=[-1, height, width, channels])
y = tf.placeholder(tf.int32, shape=[None], name="y")
labels=tf.placeholder(tf.int32, shape=[batch_size], name="labels")

conv1 = tf.layers.conv2d(inputs=X_reshaped,filters=32,kernel_size=[5,
5],padding="same",activation=tf.nn.relu)

pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)
#need to change pool size and strides according to image pixels

conv2 = tf.layers.conv2d(inputs=pool1,filters=64,kernel_size=[5,
5],padding="same",activation=tf.nn.relu)

pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2],
strides=2)#need to change pool size and strides according to image pixels

pool2_flat = tf.reshape(pool2, [-1, 15 * 18 * 64]) #change it according to
the pooling layers size:[batch_size,18432]

dense = tf.layers.dense(inputs=pool2_flat, units=1024,
activation=tf.nn.relu)

# Add dropout operation; 0.6 probability that element will be kept
dropout = tf.layers.dropout(inputs=dense, rate=0.4)

# Logits layer
# Input Tensor Shape: [batch_size, 1024]
# Output Tensor Shape: [batch_size, 50]
logits = tf.layers.dense(inputs=dropout, units=50) #size=[batch_size,50]

predictions = {
  # Generate predictions (for PREDICT and EVAL mode)
  "classes": tf.argmax(input=logits, axis=1),
  # Add `sigmoid_tensor` to the graph. It is used for PREDICT and by the
  # `logging_hook`.
  "probabilities": tf.nn.sigmoid(logits, name="sigmoid_tensor")}

 onehot_labels = tf.one_hot(indices=tf.cast(labels, tf.int32), depth=50)
 #loss = tf.losses.sigmoid_cross_entropy(multi_class_labels=labels,
 logits=logits)
 loss = tf.losses.sigmoid_cross_entropy(onehot_labels, logits=logits)

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
train_op = optimizer.minimize(loss=loss,
global_step=tf.train.get_global_step())

# Initializing the variables
init = tf.global_variables_initializer()

with tf.Session() as sess:
     sess.run(init)
     for epoch in range(n_epochs):
     for iteration in range(len(train_X) // batch_size):
          # this cycle is for dividing step by step the heavy work of each neuron
          X_batch = train_X[iteration * batch_size:iteration * batch_size + batch_size, 1:]
          y_batch = train_X[iteration * batch_size:iteration * batch_size + batch_size, 0]
          sess.run(optimizer,feed_dict={X:X_batch, y:y_batch})
     sess.run(train_op, feed_dict={X: X_batch, y: y_batch})
     acc_train =accuracy.eval(feed_dict={X: X_batch, y: y_batch})
     acc_test  =accuracy.eval(feed_dict={X: test_images, y: test_labels})
     print("Epoch:", epoch + 1, "Train accuracy:", acc_train, "test accuracy:", acc_test)
     print("Optimization Finished!")

in addition i dont understand why python takes my data as 99,999 and not 100,000.
thank you!!!

Asking these kinds of questions in the blog comments or on stack-overflow will be the best. Let's keep the bugs of this repository related to the code in the repository.


Added Contributors Profile Pics

Added Contributors Profile Pics to Appreciate their Contributions to this repository & to Encourage the younger generation to join open-source.

Thank you @AnshSinghSonkhia for contribution


bug: Allow raising Abort without providing a message

Environment information
0.1.0b4
macOS
python 3.9.7
What went wrong?
Right now, if you do:
raise Abort()

you get error:
TypeError: __init__() missing 1 required positional argument: 'message'

It would be nice if it defaulted to something generic so I can use in scripts without having to care. Like:
ERROR: Operation aborted.

How can it be fixed?
Default argument

I think it would be nice that it contains some sort of info of what triggers the aborts, like file and line no

I think it would be nice that it contains some sort of info of what triggers the aborts, like file and line no

Since the exception is not on the stack at this point, the only way I know how to do that is to inspect the call frames