[Angular] Add directive for rich texts which allows routerlinks that can be both internal and external

Add directive for rich texts which allows routerlinks that can be both internal and external.
This allows Content Editors to add internal links to other pages, while still being able to maintain external links.
Recreated from https://github.com/Sitecore/jss/pull/219 after repository maintenance.
Also add a small change so linkfield classes will be merged instead of replaced.
Description
Add directive for rich texts which allows routerlinks that can be both internal and external.
This allows Content Editors to add internal links to other pages, while still being able to maintain external links.
Motivation
Currently the implementation of a rich text is limited. All links won't use angular routerlinks.. This directive decides whether the link has a protocol and is therefor an external link.
How Has This Been Tested?
Added unit tests and tested it on our application.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have read the Contributing guide.
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.

@jlast Can you remove changes related to code formatting? Seems that some formatter is installed in your workspace.
It's hard to analyse your changes.
For example, check all places:

@jlast Can you remove changes related to code formatting? Seems that some formatter is installed in your workspace.
It's hard to analyse your changes.
For example, check all places:

Thanks for the reply, updated the merge request

@jlast I want to ask you, switch target branch to: release/15.0.0. And pull it

@jlast I'm not sure that understand correctly, but why can't we extend rich-text.directive.ts functionality in updateView function? Instead of it we have two directives for RichText, and they have difference only in terms of handling of routerlinks.
For example we have GenericLinkDirective and it extends LinkDirective, but now we have GenericRichTextDirective and it's not extending default RichTextDirective. What is the difference between Generic and default directives? I'm not in a context a little bit

@jlast I'm not sure that understand correctly, but why can't we extend rich-text.directive.ts functionality in updateView function? Instead of it we have two directives for RichText, and they have difference only in terms of handling of routerlinks.
For example we have GenericLinkDirective and it extends LinkDirective, but now we have GenericRichTextDirective and it's not extending default RichTextDirective. Maybe GenericRichTextDirective should extend RichTextDirective?
So scGenericLink - handle both internal and external links, for internal links - it is using Angular routerlinks, for external -
scLink - handle both internal and external links, for both types it's using default ? Am I right?
So you want to have same behavior but in rich-text field? Because for now RichTextDirective can't render routerlinks

I'll update this merge request probably in 1-2 weeks, working on other priorities for our client at the moment.

This will be awesome when it goes in as I was thinking of doing something similiar

@jlast I'm not sure that understand correctly, but why can't we extend rich-text.directive.ts functionality in updateView function? Instead of it we have two directives for RichText, and they have difference only in terms of handling of routerlinks.
For example we have GenericLinkDirective and it extends LinkDirective, but now we have GenericRichTextDirective and it's not extending default RichTextDirective. Maybe GenericRichTextDirective should extend RichTextDirective?
So scGenericLink - handle both internal and external links, for internal links - it is using Angular routerlinks, for external -
scLink - handle both internal and external links, for both types it's using default ? Am I right?
So you want to have same behavior but in rich-text field? Because for now RichTextDirective can't render routerlinks

@sc-illiakovalenko
To be honest I was not sure if this is what all consumers expect and use. If you feel like this can replace the current RichText implementation I would gladly replace it. Same can be said for the generic-link directive.
Please advice me how to move forward

@jlast Related PR is merged, I close this one


Moving to @snyk

Also removed old versions from the doc since they can be dropped and I do not want to search and replace in these old docs 😛

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


dev-cmd/audit: enable --[no-]signing flag for audits

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

Enables the --[no-]signing flag for auditing casks.
Should fix the error on - https://github.com/Homebrew/homebrew-cask/pull/137074

Review period will end on 2022-12-07 at 00:02:12 UTC.

Thanks @bevanjkay!

Review period skipped due to critical label.


Fix #5 Support CocoaPods

#5
Could you do pod trunk push?

Add podspec
Fix warnings 'public' modifier is redundant for static property declared in a public extension

Thanks for the pull request 👍

Could you do pod trunk push?

I've pushed NativePopup 👌
https://cocoapods.org/pods/NativePopup


qmediathekview: init at 2017-04-16

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build qmediathekview

Success on x86_64-linux (full log)
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16
shrinking /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16/bin/QMediathekView
strip is /nix/store/5qj61lcvzlap87rf6blvf8p577d482bv-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16/bin
patching script interpreter paths in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16
checking for references to /tmp/nix-build-QMediathekView-2017-04-16.drv-0 in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16...
postPatchMkspecs
postPatchMkspecs
postPatchMkspecs
/nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16

Failure on aarch64-linux (full log)
Partial log (click to expand)
make: *** [Makefile:522: moc_settings.o] Error 1
g++: fatal error: no input files
compilation terminated.
make: *** [Makefile:525: moc_database.o] Error 1
g++: error: moc_application.cpp: No such file or directory
g++: fatal error: no input files
compilation terminated.
make: *** [Makefile:543: moc_application.o] Error 1
builder for '/nix/store/z064bqkj54dysi6r5cf9n9r78k5dm1cj-QMediathekView-2017-04-16.drv' failed with exit code 2
error: build of '/nix/store/z064bqkj54dysi6r5cf9n9r78k5dm1cj-QMediathekView-2017-04-16.drv' failed

Success on x86_64-linux (full log)
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16
shrinking /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16/bin/QMediathekView
strip is /nix/store/5qj61lcvzlap87rf6blvf8p577d482bv-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16/bin
patching script interpreter paths in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16
checking for references to /tmp/nix-build-QMediathekView-2017-04-16.drv-0 in /nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16...
postPatchMkspecs
postPatchMkspecs
postPatchMkspecs
/nix/store/7w7yabm1dkhjcp7m1vi6nx1qkiiw8ch7-QMediathekView-2017-04-16

Failure on aarch64-linux (full log)
Partial log (click to expand)
make: *** [Makefile:540: moc_settingsdialog.o] Error 1
g++: fatal error: no input files
compilation terminated.
make: *** [Makefile:543: moc_application.o] Error 1
g++: error: moc_mainwindow.cpp: No such file or directory
g++: fatal error: no input files
compilation terminated.
make: *** [Makefile:534: moc_mainwindow.o] Error 1
builder for '/nix/store/z064bqkj54dysi6r5cf9n9r78k5dm1cj-QMediathekView-2017-04-16.drv' failed with exit code 2
error: build of '/nix/store/z064bqkj54dysi6r5cf9n9r78k5dm1cj-QMediathekView-2017-04-16.drv' failed

I am not sure what the problem with aarch64 is about. moc seems to be missing at the beginning of some commands. Is there a general problem with qmake or moc on aarch64?

Maybe @dezgeg can help? But if not, I'm also okay with setting the platform to x86 only.

Success on x86_64-linux (full log)
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/7jkxidrrfxg2927l2j2zkd9i3jm2044x-QMediathekView-2017-04-16
shrinking /nix/store/7jkxidrrfxg2927l2j2zkd9i3jm2044x-QMediathekView-2017-04-16/bin/QMediathekView
strip is /nix/store/5qj61lcvzlap87rf6blvf8p577d482bv-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/7jkxidrrfxg2927l2j2zkd9i3jm2044x-QMediathekView-2017-04-16/bin
patching script interpreter paths in /nix/store/7jkxidrrfxg2927l2j2zkd9i3jm2044x-QMediathekView-2017-04-16
checking for references to /tmp/nix-build-QMediathekView-2017-04-16.drv-0 in /nix/store/7jkxidrrfxg2927l2j2zkd9i3jm2044x-QMediathekView-2017-04-16...
postPatchMkspecs
postPatchMkspecs
postPatchMkspecs
/nix/store/7jkxidrrfxg2927l2j2zkd9i3jm2044x-QMediathekView-2017-04-16

Failure on aarch64-linux (full log)
Partial log (click to expand)
Package ‘QMediathekView-2017-04-16’ in /var/lib/gc-of-borg/nix-test-rs-1/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-1/pkgs/applications/video/qmediathekview/default.nix:27 is not supported on ‘aarch64-linux’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Failure on x86_64-linux (full log)
Partial log (click to expand)
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 283 ms
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 642 ms
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 1001 ms
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 2345 ms
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 5581 ms
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 9423 ms
warning: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7); retrying in 21291 ms
error: unable to download 'http://<IP_ADDRESS>:32062/nix-cache-info': Couldn't connect to server (7)


extend API to allow file extractions using md5

Extend the API to allow file extraction using MD5 hash value.
Pass an MD5 hash value as a request and return the file metadata with raw file

Hi, I would like to start contributing to this project and this issue seems interesting. Could someone please confirm that this feature should be implemented and give me some pointers on how to do so? Thank you

Added in 1.6.0


NetStandard dependency

After Upgrading to 1.8.1 (from 1.7) in R-Addin I've encountered the problem that when delivering the package to other users, the process fails to "load assembly NETSTANDARD from resources" (this is the output I get from ExcelDNA).
The Dlls I deliver along with my Addin's code are: DynamicInterop.dll, RDotNet.dll and RDotNet.NativeLibrary.dll
Did I miss anything here? I'm targeting 4.7.2 .NET in my project.
-regards,
Roland

Just to confirm, these are all of the files that you include in the RAddin folder on a non-development machine?

Well actually it’s just the three dlls packed in the Raddin-Addin-packed.xll and the .config (+ the .pdb if I want to debug). With version 1.7 I‘ve included the RDotNet.Native.dll which is not needed in 1.8.1
Regards,
Roland

Am 07.08.2019 um 19:53 schrieb Luke R<EMAIL_ADDRESS>Just to confirm, these are all of the files that you include in the RAddin folder on a non-development machine?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

My misunderstanding!  I got the add-in to load with the packed + config.  I've been trying to reproduce this, but haven't been able to.  You may have seen this thread about the issue, which towards the end makes it sound like it's a non-issue if you're using .NET 4.7.2 (which you are).  Some in the thread indicated they explicitly included NetStandard as a solution, however I've never done this in my projects.
Curious what version of Visual Studio you're using to build.  Also, when you deploy this do you know what version of .NET the client is running?  Have you been able to replicate this problem yourself?

I'm building with VS 2019 (however started development with VS2010). There is a possibility now to start projects using NETSTANDARD as targets, but this leads to other problems (with ExcelDNA). I'd rather keep targeting .NET 4.x.
Coming to a "non issue": actually this error fails to load the dlls during startup of the Addin, so the whole Addin fails therefore (also functions that are just using Modules that use R.NET without directly accessing R.NET do not work). I'd really like to upgrade, but currently it's not possible. I suspect that on my machine the netstandard dlls are somewhere distributed and on the others not. I was wondering whether these dlls could be distributed together with the addin....

Thanks for the additional info.  I had tried a few situations but couldn't reproduce yet.  I think you could explicitly include and release those DLLs with the add-in.  If you have a user experiencing the problem hopefully it's an easy enough test to see if it fixes it.


HWP-371: Comment flagging

Description
This PR includes the following proposed change(s):

Added comment flagging
View, flag and unflag

Requires
Add 'Requires Attention' label if appropriate.

[ ] Requires an update to the .env file
[ ] Requires adding a file listed in the .gitignore
[ ] Requires an opinion or answer to a question, see comments.
[ ] Requires other attention, see below.

If there are any requirements for the developer to make after this PR is merged please list them here.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Refactoring / Documentation
[ ] Version change

if your change is a breaking change, please add breaking change label to this PR
How Has This Been Tested?
Tested in swagger. Created a comment, set a flag, viewed the flags, unset the flag, and then viewed the flags.
For pull requests that are not just a simple fix, please check that the branch works on your local machine.

[ ] Tested by Zach.
[x] Tested by Brandon.
[ ] Tested by Brady.

Does the change impact or break the Docker build?

[ ] Yes
[x] No

If Yes: Has Docker been updated accordingly?

[ ] Yes
[ ] No

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[ ] New and existing unit tests pass locally with my changes
[ ] If attention is required by the developer such as updating the .env file, these requirements have been listed.

Tested using Swagger. Appears to work as intended.


Is There An Easier Way To Link Multiple Values?

Hey there,
I am trying to go through and map switch up/down status but I am having to create these manually for each port...... mostly because I don't know what the appropriate way would be to link setup these up automatically..... does anyone have any quick tips?
There's a screenshot of what I am doing now:

I would love it if I could somehow set the objects up as the values.... just looking for any advice.

Hi,
At this time, no way.
The query table is managed by grafana, not by flowcharting
sorry
BR
Arnaud


[Condition] Smooth out condition syntax

Currently:
Child: {
  $: '$condition',
  $condition: {
    val (state) {
      const query = state.getRoot().query.compute()
      if (!query) {
        return true
      } else {
        const title = state.title && state.title.compute()
        return title && title.toLowerCase().indexOf(query.toLowerCase()) !== -1
      }
    },
    $subs: {
      title: {},
      $root: { query: {} }
    }
  }
}

especialy renaming the $subs field since its completely unclear what this means -- see https://github.com/vigour-io/state/issues/93

Will be:
// no subs needed for test/condition
Child: {
  $: 'something.$test',
  $test: (state) => state.compute() === 'yes!'
}

and
// test needs subs
Child: {
  $: 'something.$test',
  $test: {
    val: (state) => state.compute() === 'yes!',
    $: {
      title: {},
      $root: { query: {} }
    }
  }
}

Implemented


Multiprocessing fails with mujoco image observations

Hi, I am trying to run the code on the mujoco Reacher-v2 env after having modified the _get_obs() function to output an image instead of joint angles:
def _get_obs(self):
    frame = self.sim.render(width=224, height=224, mode='offscreen', camera_name=None, device_id=0)
    return frame.astype(np.float32)

When I run the code using num-processes > 1, the execution freezes, until I forcefully terminate the program. It seems to hang at this point:

$ python main.py --env-name "Reacher-v2" --algo ppo --use-gae --log-interval 1 --num-steps 2048 --num-processes 4 --lr 3e-4 --entropy-coef 0 --value-loss-coef 0.5 --ppo-epoch 10 --num-mini-batch 32 --gamma 0.99 --gae-lambda 0.95 --num-env-steps 1000000 --use-linear-lr-decay --use-proper-time-limits
Logging to /tmp/openai-2019-12-23-00-13-17-486602
Creating dummy env object to get spaces
^CTraceback (most recent call last):
File "main.py", line 227, in
main()
File "main.py", line 123, in main
obs = envs.reset()
File "a2c_ppo_acktr/a2c_ppo_acktr/envs.py", line 172, in reset
obs = self.venv.reset()
File "baselines/baselines/common/vec_env/shmem_vec_env.py", line 67, in reset
return self._decode_obses([pipe.recv() for pipe in self.parent_pipes])
File "baselines/baselines/common/vec_env/shmem_vec_env.py", line 67, in
return self._decode_obses([pipe.recv() for pipe in self.parent_pipes])
File "miniconda3/envs/pytorch-1.2/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "miniconda3/envs/pytorch-1.2/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "miniconda3/envs/pytorch-1.2/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
KeyboardInterrupt

Note that when I set num-processes=1, the code runs fine. num-processes>1 works for mujoco joint angle inputs and atari image inputs, but not mujoco image inputs. What could be causing this behavior? Do I have to use a different rendering function in mujoco?

Replacing ShmemVecEnv with SubprocVecEnv solves the issue. Turns out shared memory was causing problems for mujoco rendering.


Door sensor failing to show proper state

I just upgraded from the old HomeBridge command line kit to the new integrated plugin.  Prior to the upgrade I had doors working just fine, but I had to add the following to my config:
"treatAsDoorIds": [ "242319157" , "1528402951", "408575628", "1459042103","1296285109" ],
Under the new config I simply selected each sensor was a door, but I am not seeing the door status update.  They are all stuck on "closed" in homekit.
Expected Behavior
Insteon open/close sensor state should be mapped to HomeKit
Current Behavior
No changes in state are being reflected
Steps to Reproduce (for bugs)
Open door, state updates in Insteon v7.2, but not in HomeKit
Versions

Plugin: v1.0.2

Configuration
    "platforms": [
            {
                    "protocol": "http",
                    "name": "HomeKit Bridge Server",
                    "listenPort": "8445",
                    "platform": "Indigo2",
                    "host": "<IP_ADDRESS>",
                    "debug": false,
                    "serverId": REMOVED,
                    "port": "8558"
            }
    ],
    "bridge": {
            "username": "REMOVED",
            "name": "HomeBridge",
            "pin": "XXX-YY-ZZZ",
            "port": "51826"
    },
    "accessories": [],
    "description": "HomeKit configuration generated by HomeKit Bridge on 2018-10-28 13:21:08.655000 for device HomeBridge"

}
Homebridge Log / Command Output
Log shows no events for Storage Room Door, but does show initialization:
[2018-10-28 13:21:09] Registering platform 'homebridge-camera-ffmpeg.Camera-ffmpeg'
[2018-10-28 13:21:09] ---
[2018-10-28 13:21:09] Loaded plugin: homebridge-indigo2
[2018-10-28 13:21:09] Registering platform 'homebridge-indigo2.Indigo2'
[2018-10-28 13:21:09] ---
[2018-10-28 13:21:09] Loaded config.json with 0 accessories and 1 platforms.
[2018-10-28 13:21:09] ---
[2018-10-28 13:21:09] Loading 1 platforms...
[2018-10-28 13:21:09] [HomeKit Bridge Server] Initializing Indigo2 platform...
[2018-10-28 13:21:09] [HomeKit Bridge Server] HomeKit Bridge base URL is http://<IP_ADDRESS>:8558
[2018-10-28 13:21:09] [HomeKit Bridge Server] HomeKit Bridge serverId is REMOVED
[2018-10-28 13:21:09] Loading 0 accessories...
Load homebridge-camera-ffmpeg.Camera-ffmpeg
[2018-10-28 13:21:09] [HomeKit Bridge Server] Listening on port 8445
[2018-10-28 13:21:09] [HomeKit Bridge Server] HomeKit Bridge request: http://<IP_ADDRESS>:8558/HomeKit?cmd=deviceList&serverId=REMOVED
[2018-10-28 13:21:09] [HomeKit Bridge Server] Discovered Device Door (ID 921b42a5d1546fc4958caca77a073c894aded70726d7073052b70eab9bb1a05d): Storage Room Door
[2018-10-28 13:21:09] [HomeKit Bridge Server] Initializing platform accessory 'Storage Room Door'...
Device Simulation Report
[code]
##### DEVICE DATA DUMP #####

address : 14.8E.9A
allowOnStateChange : True
allowSensorValueChange : False
batteryLevel : None
buttonGroupCount : 2
configured : True
description : Entry door to storage room

deviceTypeId :
displayStateId : onOffState
displayStateImageSel : SensorOn
displayStateValRaw : True
displayStateValUi : on
enabled : True
energyAccumBaseTime : None
energyAccumTimeDelta : None
energyAccumTotal : None
energyCurLevel : None
errorState :
folderId :<PHONE_NUMBER>
globalProps : MetaProps : (dict)
id : 242319157
lastChanged : 2018-10-28 13:03:05
lastSuccessfulComm : 2018-10-28 13:03:05
model : Open/Close Sensor
name : Storage Room Door
onState : True
ownerProps : emptyDict : (dict)
pluginId :
pluginProps : com.eps.indigoplugin.homekit-bridge : (dict)
protocol : Insteon
remoteDisplay : True
sensorValue : None
states : States : (dict)
     onOffState : on (on/off bool)
subModel :
supportsAllLightsOnOff : False
supportsAllOff : False
supportsOnState : True
supportsSensorValue : False
supportsStatusRequest : False
version : 52

##### DEVICE SIMULATION DATA #####

Service :
	alias : Storage Room Door
	model :
	subModel :
	indigoType : indigo.SensorDevice
	pluginType : Built-In
	type : Door
	desc : Door
	objId : 242319157
	serverId : 35246918
	invertOnState : False
	convertFahrenheit : False
	required : (List)
		CurrentPosition : 0
		TargetPosition : 0
		PositionState : 2
	optional : (List)
		HoldPosition
		ObstructionDetected
		Name
	native : True
	requiresPlugin : (List)
	actions : (List)
	loadOptional : False
	characterDict : (Dict)
		PositionState : 2

[/code]

I believe this may be related to my other ticket regarding the garage door as well.  I thought at first it was just the garage door, but while troubleshooting found all doors seem affected.


⚠ Bofrost.be has degraded performance

In 938267c, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1048 ms

Resolved: Bofrost.be performance has improved in 47f6577.


⚠ ClamAV has degraded performance

In 1cf6004, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 179 ms

Resolved: ClamAV performance has improved in 9f99021.


unable to disable Maven Central repository

we run dependabot in an isolated VM in azure. This VM doesn't have access to the internet but has access to a private maven repository (nexus) and azure devops. Even if we inject DEPENDABOT_EXTRA_CREDENTIALS with nexus credentials, dependabot tries to establish a connection to https://repo.maven.apache.org/maven2/ and fails.
Is there a solution to disable requests to maven.apache.org ?
Thanks for your help.

@spyrolabs I am not aware if such is available for maven as is the case with NuGet. It may be worth checking out the main dependabot repository (https://github.com/dependabot/dependabot-core).


[Feature request] Auth

Hey, first of all I'd like to thank you for making this - it's great.
I wanted to ask if it's possible to add authentication to this proxy? Like username & password.

I added .


Problem with Laravel integration

FatalThrowableError in Dialogs.php line 36:
Type error: Argument 2 passed to BotDialogs\Dialogs::__construct() must be an instance of Predis\Client, instance of Illuminate\Redis\RedisManager given, called in /path/to/project/vendor/zarincheg/telegram-bot-dialogs/src/Laravel/DialogsServiceProvider.php on line 31

P.S. You forget about DialogsServiceProvider in documentation...

@zvermafia Thank you for the report. It is a bug that happened after refactoring. I'll fix it soon.


HTC update

https://csc-guide-preview.rahtiapp.fi/origin/ktiits-workflows/computing/running/throughput/

Added links to NextFlow and Snakemake apps pages, the tutorial links added to these pages.
+
Changes as agreed in meeting with Laxman, Sami and Antoni.
Graph:
Remove: Serial or parallel subtasks?
Move Array and GNU Parallel under No answer of dependecies
Kylli
A qualitative overview of the features and capabilities of some of the workflow tools recommended by CSC is presented below. -> HTC tools
Table:
Packs jobs/job steps
Snakemak Yes
GNU Parallel NA
Array job No
Dependency support
Array jobs No
Container support -> Automatic container integration
The workflow tool supports execution with containers -> The workflow tool supports automatic execution of containers without any extra work.
Array jobs - No
GNU Parallel - No
error recovery
snakemake - Yes
Parallelization support	-> MPI/OpenMP-parallel subtasks
The workflow tool supports MPI/OpenMP-parallel subtasks
New row: multi-partition support
Support
NextFlow, Snakemake: pay attention
Array jobs, GNU: no
Fireworks, HyperQueue: ??

@samiilvonen promised to see also the srun note, because that is rather unclear text now.

Hi! This looks great! Please also consider, once ready, adding a whatsnew entry and the link also in the tutorials section!

In tutorials, there already is/was a link.
But with whatsnew I am not familiar, how to add stuff there. Could somebody else do that? Or I do not know if it is even worth adding there, the changes here are still rather small scale?

It's is worth adding what's new. It's super easy. See instructions here: https://github.com/CSCfi/csc-user-guide/blob/master/FAQ.md#how-do-i-add-an-entry-to-the-whats-new-section
When it's here, it gets shown on the landing page and mentioned in the next newsletter.

Laxmana is doing updates also to Snakemake page in another PR. I would suggest that we do one common What's new for both of these PRs.

Thanks, Kylli. Flowchart and table look bit more clear now.  Could we add a sentence or two to describe flowchart? And also links provided in the flowchart are broken with an exception to snakemake. Currently, snakemake URL points to external documentation while others to our CSC docs. Not sure if these broken links will be resolved in production.


December 2023

Pushed December changes to main branch

December month update.


'gallery_saver/gallery_saver-Swift.h' file not found

I added gallery_saver to my project, and got an error as below when building iOS app.
** BUILD FAILED **
Xcode's output:
↳
=== BUILD TARGET sqflite OF PROJECT Pods WITH CONFIGURATION Debug ===
/Users/Ranyzhang/WorkSpace/Flutter/flutter/.pub-cache/hosted/pub.dartlang.org/gallery_saver-1.0.1/ios/Classes/GallerySaverPlugin.m:2:9: fatal error: 'gallery_saver/gallery_saver-Swift.h' file not found
#import <gallery_saver/gallery_saver-Swift.h>
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.`

Try to run "pod install" in console.

The same error.
pod install did not help

@idrats @rathinho Since Flutter does not support Swift by default, you'll have to create your project
by using this command " flutter create -i swift my_app" in order to make this plugin works for iOS.

@idrats @rathinho is this still happening with the latest version of gallery saver?

Actually I did not remember what kind of actions helped me to solve this problem, but now I can not check if concrete this problem is solved. For me it is working

Here is the solution, first you need to make sure to do what @ailic88 said, your project needs swift: https://stackoverflow.com/a/56281718
Once that is done, this is what else I noticed: this error only pops up when I do the following and start with a fresh repo:
git clean -xdf
<Launch VSCode or anything else and try to develop>

This never worked for me. So I have to make sure that I insert one extra step before I get started on development (one time):
git clean -xdf
flutter build ios
<Launch VSCode or anything else and try to develop>

This always works. Juts have to call flutter build ios (from command line) first which apparently triggers a pod install a little differently and  makes sure that those swift files get built in the right location.
Haven't looked into it enough to understand why, but this should get you passed the error and on to the fun stop... slinging code.

@ailic88 Idk if you are the main developer but just thought I would pass on this data point. It was not clearly obvious this is what I had to do, nor is your comment in https://github.com/CarnegieTechnologies/gallery_saver/issues/13#issuecomment-521609402  - the docs were not that helpful either form a "set up" perspective:
https://pub.dev/documentation/gallery_saver

This is a good package and works exactly as expected once set up!

@walsha2  Usually when this error occurred problem is that user didn't
add use_frameworks! in Podfile or environment is not setup properly.
Just like you've written above few cleaning fix the problem :).
Anyway thanks for feedback, we will improve our documentation.


pass on parent env vars with values

The variables specified in ParentEnv weren't extracted from the running environment.

makes sense

But the issue with it that caching that ParentEnv is intended for disappears. Idea here is to ask OS env vars once and then reuse them on each program fork.... Why it does not work for you?

Sorry for the delay in response.
The value for the parentEnv environment variables are not passed to the subprocess.
If I specify a "VCAP_APPLICATION" value in parentEnv then the current code will append "VCAP_APPLICATION" to the env array without including the value.
I put fmt.Println(env) in the function launchCmd, this is what the output looks like with the original code
  GATEWAY_INTERFACE=websocketd-CGI/0.1
  VCAP_APPLICATION
  SERVER_PROTOCOL=HTTP/1.1
  ...

After the subprocess is started by libwebsocketd the environment variable VCAP_APPLICATION is not visible from within the process.
With this pull request the variable name and the value is appended to the env array and fmt.Println(env) outputs the following:
  GATEWAY_INTERFACE=websocketd-CGI/0.1
  VCAP_APPLICATION={"limits":{"mem":64,"disk":1024,"fds":16384}}
  SERVER_PROTOCOL=HTTP/1.1
  ...

This way the VCAP_APPLICATION parentEnv is properly passed on to the subprocess.

Could you send your results? What OS you are using? What Go version?
Here is my test:
$ MYVAR=11111111111111 websocketd --port 8080 --devconsole --loglevel deg --passenv MYVAR env
Tue, 24 Mar 2015 08:20:29 -0400 | INFO   | server     |  | Serving using application   : /usr/bin/env
Tue, 24 Mar 2015 08:20:29 -0400 | INFO   | server     |  | Starting WebSocket server   : ws://:8080/
Tue, 24 Mar 2015 08:20:29 -0400 | INFO   | server     |  | Developer console enabled   : http://:8080/
Tue, 24 Mar 2015 08:20:41 -0400 | ACCESS | http       | url:'http://localhost:8080/' | DEVCONSOLE
Tue, 24 Mar 2015 08:20:46 -0400 | DEBUG  | env        | url:'http://localhost:8080/' id:'1427199646099298214' remote:'localhost.localdomain' command:'/usr/bin/env' | Std. variable: SERVER_SOFTWARE=websocketd/0.2.10
Tue, 24 Mar 2015 08:20:46 -0400 | DEBUG  | env        | url:'http://localhost:8080/' id:'1427199646099298214' remote:'localhost.localdomain' command:'/usr/bin/env' | Parent envvar: MYVAR=11111111111111
......

Everything looks as expected, MYVAR is passed.

I think I figured out what I was doing wrong. In libwebsocketd this is the Config struct I use:
var websocketdConfig = &libwebsocketd.Config{
	CommandName: "bash",
	DevConsole:  true,
	Ssl:         true,
	ParentEnv:   []string{"VCAP_APPLICATION"},
}

After digging in the websocketd codebase for a bit I saw that the passenv command line flag to websocketd actually passes on a "%v=%v" key/value pair.
Just to make it clear: the solution was to run this code before creating the websocketd server:
for i := 0; i < len(websocketdConfig.ParentEnv); i++ {
  key := websocketdConfig.ParentEnv[i]
  websocketdConfig.ParentEnv[i] = fmt.Sprintf("%v=%v", key, os.Getenv(key))
}

I find it counter-intuitive. I expected the app to pass on the environment variables with their actual contents after specifying the list in websocketdConfig.ParentEnv i.e. the same behaviour as the passenv flag

Hi @cromega
Although we make a lot of effort to keep websocketd (the program) stable, we're not quite at the stage where libwebsocketd (the API) is stable. I just wanted to warn you as we frequently make internal changes which may break if you're using the API.


ui: improve ux for backup code flow. fixes #332 WD-16907

Done

fix setup backup code screen ui
fix confirm backup code screen ui
add confirmation modal for removing backup codes
adjust copy in all backup code related screens

Screenshots

Resolved the comments and addressed all suggestions. Also refactored the NodeInputSubmit to be clean of any use case specific logic.

Thanks for the changes, the logic is nicely separated now :+1: just a few small nits left

@mas-who @BarcoMasile can you have another pass at this please?


New version: CoordinateDescent v0.2.0

Registering package: CoordinateDescent
Repository: https://github.com/mlakolar/CoordinateDescent.jl
Created by: @mlakolar
Version: v0.2.0
Commit: 2af7191b56cc69d22730a54567e484bdd0c7e00c
Git reference: master

Please add (at least)
[compat]
julia = "1"

to your project file and retrigger the bot.

@mlakolar ^


store meta information

I need to store if one is an archive's owner. What about removing the name property and instead allowing for a map of arbitrary meta information to be associated with a drive?

Solved in #8 :sparkles:


Update githubbot -> octobot for paths

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

sandbox/TOC.yml
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/azure-config.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/azure-deploy.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/bot-keys.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/demo.gif
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/github-keys.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/import.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/index.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/luis-appid.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/luis-key.png
:white_check_mark:Succeeded
View

sandbox/demos/media/octobot/luis-publish.png
:white_check_mark:Succeeded
View

sandbox/demos/octobot.md
:white_check_mark:Succeeded
View

sandbox/index.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Final objects

Working towards 2.x I feel like asking..
Should some objects become final?

I vote yes,
But I know that many users would hate that....
I introduced final objects in nyholm/psr7


Issue #52 - add Engine support for CQL external functions

This is an initial working implementation of CQL external functions in the engine.
It was co-developed by Darren and Chris.

Please hold off on merging this.  I discovered it added a bug.  A fix is pending shortly.

The current external functions implementation works at the whole engine level, operating by adding extra functions as if they were built-in to the engine and available for all libraries the engine powers.  The Java code instantiating the engine Context object names the Java library defining the external functions, and the CQL libraries themselves don't know anything about that, they only know they can use the extra functions.  Altering this system so the CQL libraries can be aware of multiple external function libraries should be left to another task, and the current work should be merged as is.  Such changes will take their own rounds of planning and deciding what works best.

The external function provider itself is not associated with any specific CQL library.  It is only meant to be a resource that one or more CQL libraries can use, same as they use e.g. FHIRHelpers now.  They just happen to not be implemented in CQL but rather in Java.

As Bryn also stated, I am fine with merging this work as is and creating a new ticket for his suggestion. @duncand I am not sure what you mean by "Altering this system so the CQL libraries can be aware of multiple external function libraries". Why would the CQL need to be aware of multiple external function libraries? It should be as simple as registering external function providers (EFP) using a map with the CQL library id as the key. Then when an external function provider is retrieved, just use the current library stack to retrieve the correct EFP.

@c-schuler - On further thought I think I get what is being suggested now.  I will make an alteration to the code so that an external provider library is mapped with a CQL library via the declared name of the latter.  I agree this is a better design.

@c-schuler or @brynrhodes - Please review my altered version, and if you accept that, please merge it, thank you.

Looks good. One thing I would like to see is a more thorough test that tests a possible use case (like libraries that reference a function with the same signature, but have different results).
Something like the following:
library CqlExternalFunctionsTest version '3'

include AnotherLibrary version '1' called AnotherLibrary

context Patient

define function MyStringOperation(x String) returns String : external

define CallCurrentStringOperation:
    MyStringOperation('Testing') // returns TESTING

define CallAnotherStringOperation:
    AnotherLibrary.MyStringOperation('Testing') // returns testing

I am happy to merge as is if you think that request is overkill.

I think its hard to know where to draw the line on additional tests.  If we were going to add more than the ones I have then there are probably a bunch of scenarios that should be covered and just doing the one you mention seems arbitrary.  Please merge the existing one but we can make an issue to cover adding more in a broader sense.


Metadata pagination isn’t returning same results as scrolling

The issue is not consistent, but some queries return more series from Elasticsearch when using scrolls instead of search after pagination.

The problem seems to be with sorting in the ES request. The query doesn't really care about sort order, and until recently didn't even set a sort parameter. For scrolling that's fine, but it presents a problem for live pagination. The order currently appears to be nondeterministic so the subsequent searches can end up not having perfect overlap and some series can be missed. I imagine its also possible some series are duplicated.
The only unique field we have is _id and ES takes a big performance hit to do anything with _id. The recommendation is to copy it to a new field that has doc_values enabled and sort on that.

Apparently you can't modify the _id field anymore so I don't think its possible to copy it in a mapping. I'm going to try sorting on _timestamp instead (it doesn't matter what we sort on, we just need consistent results).

There does not appear to be any way to copy the _id field in a mapping (despite the documentation seemingly say to do so). So options seem to be:

generate the ID in Heroic instead of letting ES autogenerate it
Use a script sort to sort based on the tags. This should generate unique values since each document represents one set of tags and nothing else. Not sure if there are any considerations around using script sorting to be aware of.
Use _id directly. Not recommended as it will almost certainly will cause performance issues.

Another option from @malish8632: when the consumers insert the document into ES, have them add a hash of the tags as a new field. This would provide a unique field that doesn't require query scripts.


Added Metadata to ColumnInfo which can be accessible from IMapper

Hi @schotime, I would love some feedback on this!
We have builtin solution for encryption by our custom encryption mechanism. We want to map specific properties using FluentMappings but there is no possible to pass additional metadata to the IMapper.
With this update I added Metadata dictionary which can be populated in fluent mappings and can be passed to any IMapper instance.
I will be happy if you accept my pull-request. 😃

Hey, nice one. like it, however it's a breaking change.
And all the formatting changes that make it hard to decipher what you've changed, lol.
Ideally if we can find a way to make this non-breaking that would be great. I have some ideas, but would need to test them out. Happy for you to have a try though.

And all the formatting changes that make it hard to decipher what you've changed, lol.
Sorry for that. It's by habit. 😄

Ideally if we can find a way to make this non-breaking that would be great. I have some ideas, but would need to test them out.
Right. Maybe overloaded methods on IMapper will be just enough?

We actually do the same thing but we wrap all encrypted fields in a class which can then easily be mapped conventionally. Has the benefit of knowing which fields are encrypted just by looking at them.

You mean base class or just additional class which maps to special table (for encrypted data) in db?


PWX-36081 : Adding check for PodDisruptionBudget value during kubernetes upgrade

What this PR does / why we need it: This PR adds validation to the poddisruptionbudget value during kubernetes upgrade in a cluster. It constantly monitors the pdb value throughout the upgrade to ensure the value of minAvailable does not change and only nodes lesser than allowed disruptions are down.
Which issue(s) this PR fixes (optional)
Closes # PWX-36081
Special notes for your reviewer:
The code has been tested using this jenkins job where upgrade of OCP cluster is performed.

I would like to run this also on some none OCP cluster upgrades, OCP upgrades are pretty pretty straight forward, but AKS, EKS, GKE upgrades are more complex and I have seen things that been added work on OCP but not on other cloud upgrades. Let me know if you need to help to run the fix in the private image on some of these before merging, otherwise it looks good to me


Update docs link

The two links were different but get to the same place.  The docs.fermentrack.com link generated a certs error due to name mismatch (cert expects readthedoics.io domain). If you ignore the error you wind up on a 404 page.

I know little about source code control/git.  Should I submit this against the dev branch?

Yes please!
Sent from my iPhone
On Apr 4, 2019, at 12:41 PM, uSlackr <notifications<EMAIL_ADDRESS>wrote:
I know little about source code control/git. Should I submit this against the dev branch?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/thorrak/fermentrack/pull/352#issuecomment-479973852, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACWBLMcinxnntHIbrAz_PQclRe_lCwvnks5vdisfgaJpZM4cdPIM.

If you want to reject that, I’ll do it on Dev & resubmit.
\Greg

From: John<EMAIL_ADDRESS>Sent: Thursday, April 4, 2019 12:56:43 PM
To: thorrak/fermentrack
Cc: uSlackr; Author
Subject: Re: [thorrak/fermentrack] Update docs link (#352)
Yes please!
Sent from my iPhone
On Apr 4, 2019, at 12:41 PM, uSlackr <notifications<EMAIL_ADDRESS>wrote:
I know little about source code control/git. Should I submit this against the dev branch?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/thorrak/fermentrack/pull/352#issuecomment-479973852, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACWBLMcinxnntHIbrAz_PQclRe_lCwvnks5vdisfgaJpZM4cdPIM.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/thorrak/fermentrack/pull/352#issuecomment-479979333, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYjvZ_RcqzxxnuYrsgI5BK_XfwDw68vQks5vdi7LgaJpZM4cdPIM.


ChaosOrca: Cleanup

Cleans up ChaosOrca code and adds dedicated folder for experiment data. (This includes the raw csv data, evaluations and summaries.)

Merged, thanks @JSimo !


ci: package and publish the Maya plugin to the GitHub release

What was the problem/requirement? (What/Why)
The actual plugin files for the Maya submitter:

Are not having their version number modified when we release a new version of the submitter; and
Are not made available to customers other than by requiring that they git clone the release branch of this repository.

What was the solution? (How)
Modify the GitHub workflows to:

Overwrite the DeadlineCloudForMaya.mod file with the correct version during creation of the bump PR; and
Zip the contents of the maya_submitter_plugin directory into the dist_extras/ directory during the actual release workflow. The shared release workflow has been modified in the .github repository to publish all files in dist_extras/ to the GitHub release page.

What is the impact of this change?
With this change, we can have instructions for how to install the Maya submitter in the package README that looks like:

Install the submitter support files into Maya using mayapy. Reference the Maya online documentation to find the location of mayapy on your system. e.g. For Maya 2024: https://help.autodesk.com/view/MAYAUL/2024/ENU/?guid=GUID-72A245EC-CDB4-46AB-BEE0-4BBBF9791627 Follow the instructions in Maya's official online documentation to install the deadline-cloud-for-maya Python library and its dependencies to be available to Maya. For example, to install in Maya's default global location:

mayapy -m pip install deadline-cloud-for-maya --upgrade

Download the appropriate version of the maya_submitter_plugin.zip file from the GitHub release for this package.
Unzip the maya_submitter_plugin into the location of your MAYA_MODULE_PATH. See the Maya online documentation on "File path variables". e.g. For Maya 2024 https://help.autodesk.com/view/MAYAUL/2024/ENU/?guid=GUID-228CCA33-4AFE-4380-8C3D-18D23F7EAC72

How was this change tested?
I believe the only way to test it is in live. I've tested out the packaging and .mod file generation commands manually outside of the GitHub action workflow.
Did you run the "Job Bundle Output Tests"? If not, why not? If so, paste the test results here.
N/A
Was this change documented?
N/A
Is this a breaking change?
It it not intended to be so.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Note to self.
TODO: Add the instructions on how to install the maya submitter to the README in this PR

Closing. This has been taken over by Chuck in  https://github.com/aws-deadline/deadline-cloud-for-maya/pull/174


When the python pex file is copied over to the container, only base name should be reffered

We copy the pex file into the ./ directory of the container. Thus if I do
heron submit ... <path/to/my/pex/topology.pex> only topology.pex should be referenced as the topology bin

I suspect there are reasons why the SubmitterMain would want the entire path. Should we truncate it lower down in the call stack? Would this work in cases where the defn file isn't in the rood directory where the client is invoked? I'd expect the submitter needs the full path to the file right?

The submitter main actually does nothing with this argument. All the packing of the defn/bin happens inside the cli before calling the submitter. The submitter merely copies this as the arg to the executor.

Disagree with that. Infact this looks like a bug to me.
Schedulers should not be doing this truncating business. It should be done at a base layer(which is cli). Otherwise we will have the same situation like the one we have today where some schedulers handle it properly and others dont. For instance, aurora handles it fine(does a basename, and thats pretty much the only thing that it does). The others(including LocalScheduler) do not do that.
Unless there is a strong reason why schedulers would want the absolute filename, I would vote for merging this change.

I believe that comment needs to be changed. And there should be no reason why the scheduler needs fq path. Please let me know why would the scheduler would need it.

Ultimately the scheduler abstraction should be as focused as possible. It needs to get bare minimum things a) topology contents and b) parameters affecting scheduler like number of containers and container description(ram/cpu, etc). Ultimately the scheduler's main job is to just act as the interface to the actual scheduler implementation to fire off the containers and start the executor on each container. The rest should be handled outside.

So I checked our schedulers. Some of them does truncation using FileUtils.getBaseName, some of them invoke executorCommandArgs to invoke getBaseName indirectly. I think we should do truncation for all schedulers and do it once at one place only.

Great that we agree that we need to do this at a central place :)
Only debate now is where.

The submitter takes a number of (valid) local file paths and uses them to build configs to represent what will be the remote settings on the cluster. I recommend we stick with that pattern and do that transformation where we make the remote configs. That's here in SubmitterMain:
  protected static Config topologyConfigs(
      String topologyPackage, String topologyBinaryFile, String topologyDefnFile,
      TopologyAPI.Topology topology) {
    PackageType packageType = PackageType.getPackageType(topologyBinaryFile);

    return Config.newBuilder()
        .put(Key.TOPOLOGY_ID, topology.getId())
        .put(Key.TOPOLOGY_NAME, topology.getName())
        .put(Key.TOPOLOGY_DEFINITION_FILE, topologyDefnFile)
        .put(Key.TOPOLOGY_PACKAGE_FILE, topologyPackage)
        .put(Key.TOPOLOGY_BINARY_FILE, topologyBinaryFile)
        .put(Key.TOPOLOGY_PACKAGE_TYPE, packageType)
        .build();
}

https://github.com/twitter/heron/blob/master/heron/scheduler-core/src/java/com/twitter/heron/scheduler/SubmitterMain.java#L81
I don't love the inconsistency of mixing remote and local paths in topologyDefnFile, topologyPackage, topologyBinaryFile here as doing so has bit us in the past, but we can at least centralize the fix for this field here. When doing so we should remove all downstream FileUtils.getBaseName calls since we're changing the contract to where the impl already provides that now, which we should also document in Key.TOPOLOGY_BINARY_FILE.

So here is the interesting part. The cli already does a basename conversion while preparing the topology.tar.gz. See https://github.com/twitter/heron/blob/master/heron/tools/common/src/python/utils/config.py#L61
Moreoever the submitter main at any point does not require fq path of either the defn file/topology binary. How about consolidating all of that within cli and cleaning all basename stuff in all schedulers?

Also note that none of the config files are really available to the scheduler. The cli bundles all of them inside the topology.tar.gz.


Tensorflow 1.11.0 is still looking for cublas 9.0 (and other libraries) despite of having installed CUDA 10

Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Ubuntu 18.04

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:

TensorFlow installed from (source or binary):

pip3 install tensorflow-gpu==1.11.0

TensorFlow version (use command below):
Python version:

I use a local installation: /home/ivan/Python/Python-3.6.5

Bazel version (if compiling from source):

GCC/Compiler version (if compiling from source):

CUDA/cuDNN version:
CUDA 10.0 and CuDNN 7.3.1 compatible with CUDA 10.0

GPU model and memory:

NVIDIA Corporation GM107GL [Quadro K2200] with 65 GB of RAM

Exact command to reproduce:
import tensorflow as tf

It produces this error message;
Traceback (most recent call last):
File "Main_55_Lithofacies_Classification.py", line 19, in
import tensorflow as tf
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/init.py", line 22, in
from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/init.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
Collecting system information...
Traceback (most recent call last):
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/home/imarroquin/Python/Python-3.6.5/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
Tensorflow 1.11.0 is trying to load cublas from CUDA 9.0. Unfortunately, I can't install CUDA 9.0 on Ubuntu 18.04 because is not supported by Nvidia (https://devtalk.nvidia.com/default/topic/1036640/cuda-setup-and-installation/cuda-9-0-install-on-ubuntu-18-04-quot-unmet-dependencies-quot-/ )
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

CUDA 10.0 support is coming soon in a released version of Tensorflow, see https://github.com/tensorflow/tensorflow/issues/22706 for more details.
On the other hand, it might not be officially supported by NVIDIA, but it certainly possible to install CUDA 9.0 on Ubuntu 18.04. Here's proof from my system:
$ python -c 'import tensorflow as tf; print(tf.VERSION)'
1.11.0

$ python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib() + "/python/_pywrap_tensorflow_internal.so")' | xargs ldd | grep libcu
	libcublas.so.9.0 => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcublas.so.9.0 (0x00007fa334ec5000)
	libcusolver.so.9.0 => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcusolver.so.9.0 (0x00007fa3302ca000)
	libcudart.so.9.0 => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0 (0x00007fa33005d000)
	libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007fa32df33000)
	libcudnn.so.7 => /usr/lib/x86_64-linux-gnu/libcudnn.so.7 (0x00007fa31bbfa000)
	libcufft.so.9.0 => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcufft.so.9.0 (0x00007fa313b59000)
	libcurand.so.9.0 => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcurand.so.9.0 (0x00007fa30fbf5000)

$ lsb_release -d
Description:	Ubuntu 18.04.1 LTS

Thanks @jacquerie  for the quick reply. I hope that the new version will be made available soon

Here is my new tutorial for Building Tensorflow 1.12 + CUDA 10.0 + CUDNN 7.3.1 + NCCL 2.3.5 + bazel-0.17.2 https://www.python36.com/how-to-install-tensorflow-gpu-with-cuda-10-0-for-python-on-ubuntu/, you will also get prebuilt wheel at last with AVX and compute capability 5.0.

https://drive.google.com/file/d/1QV7fBi5ZpTm02N1_QbyEhT6v80B6Zffg/view?usp=sharing

@ivan-marroquin  - Is this still an issue ?

Closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

im still having this issue with tensorflow 1.12 depite having compiled from source with cuda 10

Yes it is still an issue. Tensorflow's docker images are broken: https://www.tensorflow.org/install/docker links against cuda 9, but deploys will cuda 10

The latest release TF 1.13.0-rc0 comes with cuda 10 support. Therefore the docker gpu images are deployed with cuda 10.

The latest release TF 1.13.0-rc0 comes with cuda 10 support. Therefore the docker tf gpu images are deployed with cuda 10.

Maybe google itself hasn't been updating their stuff?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

This is inside of the official nightly devel-gpu image:
ldd libtensorflow_framework.so
	linux-vdso.so.1 =>  (0x00007ffc8a15a000)
	libcublas.so.9.0 => not found
	libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007f0258749000)
	libcudnn.so.7 => /usr/lib/x86_64-linux-gnu/libcudnn.so.7 (0x00007f0243922000)
	libcufft.so.9.0 => not found
	libcurand.so.9.0 => not found
	libcudart.so.9.0 => not found
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f024371e000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0243415000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f02431f8000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0242e76000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0242c60000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0242896000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f025a7aa000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f024268e000)
	libnvidia-fatbinaryloader.so.415.27 => /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.415.27 (0x00007f0242441000)

So, clearly linked to cuda 9. BUT:
ls -la /usr/local/cuda/lib64/
total 441176
drwxr-xr-x 1 root root      4096 Feb  1 15:12 .
drwxr-xr-x 1 root root      4096 Feb  1 15:11 ..
lrwxrwxrwx 1 root root        14 Sep 12 21:37 libOpenCL.so -> libOpenCL.so.1
lrwxrwxrwx 1 root root        16 Sep 12 21:37 libOpenCL.so.1 -> libOpenCL.so.1.1
-rw-r--r-- 1 root root     27096 Sep 12 21:35 libOpenCL.so.1.1
lrwxrwxrwx 1 root root        19 Sep 12 21:37 libaccinj64.so -> libaccinj64.so.10.0
lrwxrwxrwx 1 root root        23 Sep 12 21:37 libaccinj64.so.10.0 -> libaccinj64.so.10.0.130
-rw-r--r-- 1 root root   7407024 Sep 12 21:35 libaccinj64.so.10.0.130
lrwxrwxrwx 1 root root        21 Sep 12 21:37 libcublas.so.10.0 -> libcublas.so.10.0.130
-rw-r--r-- 1 root root  70796360 Sep 12 21:35 libcublas.so.10.0.130
-rw-r--r-- 1 root root    695156 Sep 12 21:35 libcudadevrt.a
lrwxrwxrwx 1 root root        17 Sep 12 21:37 libcudart.so -> libcudart.so.10.0
lrwxrwxrwx 1 root root        21 Sep 12 21:37 libcudart.so.10.0 -> libcudart.so.10.0.130
-rw-r--r-- 1 root root    495736 Sep 12 21:35 libcudart.so.10.0.130
-rw-r--r-- 1 root root    955082 Sep 12 21:35 libcudart_static.a
lrwxrwxrwx 1 root root        20 Sep 12 21:37 libcufft.so.10.0 -> libcufft.so.10.0.145
-rw-r--r-- 1 root root 103177128 Sep 12 21:35 libcufft.so.10.0.145
lrwxrwxrwx 1 root root        21 Sep 12 21:37 libcufftw.so.10.0 -> libcufftw.so.10.0.145
-rw-r--r-- 1 root root    561192 Sep 12 21:35 libcufftw.so.10.0.145
lrwxrwxrwx 1 root root        18 Sep 12 21:37 libcuinj64.so -> libcuinj64.so.10.0
lrwxrwxrwx 1 root root        22 Sep 12 21:37 libcuinj64.so.10.0 -> libcuinj64.so.10.0.130
-rw-r--r-- 1 root root   7792472 Sep 12 21:35 libcuinj64.so.10.0.130
-rw-r--r-- 1 root root     31954 Sep 12 21:35 libculibos.a
lrwxrwxrwx 1 root root        21 Sep 12 21:37 libcurand.so.10.0 -> libcurand.so.10.0.130
-rw-r--r-- 1 root root  60806128 Sep 12 21:35 libcurand.so.10.0.130
lrwxrwxrwx 1 root root        23 Sep 12 21:37 libcusolver.so.10.0 -> libcusolver.so.10.0.130
-rw-r--r-- 1 root root 139257368 Sep 12 21:35 libcusolver.so.10.0.130
lrwxrwxrwx 1 root root        23 Sep 12 21:37 libcusparse.so.10.0 -> libcusparse.so.10.0.130
-rw-r--r-- 1 root root  59078736 Sep 12 21:35 libcusparse.so.10.0.130
lrwxrwxrwx 1 root root        18 Sep 12 21:37 libnvToolsExt.so -> libnvToolsExt.so.1
lrwxrwxrwx 1 root root        22 Sep 12 21:37 libnvToolsExt.so.1 -> libnvToolsExt.so.1.0.0
-rw-r--r-- 1 root root     37240 Sep 12 21:35 libnvToolsExt.so.1.0.0
lrwxrwxrwx 1 root root        21 Sep 12 21:37 libnvblas.so.10.0 -> libnvblas.so.10.0.130
-rw-r--r-- 1 root root    596080 Sep 12 21:35 libnvblas.so.10.0.130
drwxr-xr-x 2 root root      4096 Feb  1 15:11 stubs


Update golang packages

Modifies the SDK proto files to change the golang package name, keeping it consistent with the directory layout in /proto/.
Updates the generated proto files in go/proto/ to use the new namespaceing.
Updates the go/services/ files to use the new imports.
Changing the package names prevents collisions with services that have the same endpoints (e.g. account.info and provider.info) and allows for language tooling to auto-resolve packages.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@9ba33e6). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #710   +/-   ##
=======================================
  Coverage        ?   88.03%
=======================================
  Files           ?       53
  Lines           ?     2114
  Branches        ?        0
=======================================
  Hits            ?     1861
  Misses          ?      253
  Partials        ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ba33e6...1fcfc08. Read the comment docs.

Minor PR that aligned server with sdk protos - https://github.com/trinsic-id/sdk/pull/712


Question: Using the MMFashion segmentation model with mmdet

I am trying to use the fashion parsing segmentation model from MMFashion, by loading the pre-trained weights from https://github.com/open-mmlab/mmfashion/blob/master/docs/MODEL_ZOO.md#fashion-segmentation using the latest version of mmdet.
I tried following the instructions at: https://github.com/open-mmlab/mmdetection/tree/main/configs/legacy_1.x
I managed to correctly load the pre-trained weights with init_detector and run an image through inference_detector, but the result contains zero masks. Either my image does not contain a recognizable person/clothes, or the model is somehow loaded/executed incorrectly.

I might have made a mistake. I do get out masks now.
For future reference, I used the configuration at https://github.com/open-mmlab/mmdetection/blob/main/configs/legacy_1.x/mask-rcnn_r50_fpn_1x_coco_v1.py


[Bug]: PinInput field is white in dark mode

Reproduction
https://stackblitz.com/edit/agfptq?file=src%2FApp.vue
Describe the bug
The PinInputInput.vue component does not have a 'bg-background' class that other input components do have.
This causes the component to remain white in the dark mode.
System Info
System:
    OS: Windows 11 10.0.22631
    CPU: (12) x64 12th Gen Intel(R) Core(TM) i5-12400
    Memory: 9.43 GB / 31.78 GB
  Binaries:
    Node: 20.16.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.9.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (127.0.2651.74)
  npmPackages:
    @vueuse/core: ^11.2.0 => 11.2.0
    radix-vue: ^1.9.8 => 1.9.9
    vue: ^3.5.10 => 3.5.12

Contributes

[X] I am willing to submit a PR to fix this issue
[X] I am willing to submit a PR with failing tests

Fixed by adding this to my css:
html.dark {
    color-scheme: dark;
}

as suggested by @sadeghbarati in https://github.com/unovue/shadcn-vue/pull/895#issuecomment-2472555075


Add tests

We have no tests whatsoever for lintlizard, while it should be fairly easy to mock out subprocess and make assertions on what commands are executed (and potentially what output is generated by lintlizard).

Half of the "test suite" of lintlizard is running it on itself in CI. I agree having some actual tests would be nice though. As well as testing failures.

Well, --fix or --changed aren't running in CI (to my knowledge), so it's possible to break them without noticing

It's only checking for non-zero exit codes, right?
https://github.com/closeio/lintlizard/pull/147

Yep that's true, it's only regular runs.


[SPARK-13043][SQL] Implement remaining catalyst types in ColumnarBatch.

This includes: float, boolean, short, decimal and calendar interval.
Decimal is mapped to long or byte array depending on the size and calendar
interval is mapped to a struct of int and long.
The only remaining type is map. The schema mapping is straightforward but
we might want to revisit how we deal with this in the rest of the execution
engine.

Test build #50241 has started for PR 10961 at commit 24ca13c.

Test build #50287 has started for PR 10961 at commit ec397e3.

Test build #50287 has finished for PR 10961 at commit ec397e3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50287/
Test PASSed.

cc @davies for a more detailed review

LGTM

Going to merge this. Thanks.


Migrate Z Parameter Sweep Results from Tybalt Here

Previously, greenelab/tybalt included the analysis scripts, results, and figures for an initial parameter sweep along different z dimensionality. The results were summarized here.
The analysis best belongs in this repository. This is in the interest of keeping Tybalt clean and also consolidating the analysis toward compression feature interpretation. 99% of this code has already been reviewed in greenelab/tybalt#116 - there were some required commits to add correct paths to make analysis executable.

@dhimmel - there is no need to review the files (but feel free if you want to!). @danich1 and @jaclyn-taroni have already reviewed them as part of greenelab/tybalt
As this project evolves, @cgreene and I thought it best to migrate these analyses away from Tybalt.
As it stands, the interpret-compression directory will include a series of analyses. The initial z sweep is one of them. The hetmech applications will become one of the analyses and will also require a restructuring within the repository. Does this sound like reasonable organization to you?


Docker Script Fix

Information

QPPCT-661

Changes proposed in this PR:

Change docker_deploy.sh to return failure on an ansible failure

Checklist

[n/a] All JUnit tests pass (mvn clean verify).
[n/a] New unit tests written to cover new functionality.
[n/a] Added and updated JavaDocs for non-test classes and methods.
[n/a] No local design debt. Do you feel that something is "ugly" after your changes?
[n/a] Updated documentation (README.md, etc.) depending if the changes require it.

Can you explain the changes a bit more to the less-than-proficient shell scripters (me)?

@omnivion sure. It is checking ansible's response to see if it is a non-zero response and purposefully exiting with -1. Before, we were getting the failure response from ansible but then just ignoring it and continuing to run.


syncing conda recipe when dependencies in setup.py change

I realized that when we update dependencies in the setup.py (example), the PR that the bot opens for conda-forge does not update them - so we need to fix this.
the PR will break when new dependencies are added, removed but it'll pass when an existing dependencies is changed. e.g., from pkg>=1 and pkg>=2

looks like this is what want: https://twitter.com/condaforge/status/1647933355877445633

example:
this is the PR that the conda bot opened: https://github.com/conda-forge/jupysql-feedstock/pull/10
it didn't update the dependencies: https://github.com/ploomber/jupysql/blob/ca49215ae5bcb43822c76c7ebb22cfe430550127/setup.py#L21
cc @idomic

applied this to the jupysql recipe, will propagate to other packages if it works well: https://github.com/conda-forge/jupysql-feedstock/blob/main/conda-forge.yml


S3 upload_part creates 2 extra copies of payload/body in memory

Using S3Client when calling upload_part it will .clone() provided body Vec<u8> (since it is behind pointer to UploadPartRequest. Looking at memory usage of my client there is another copy done somewhere else.
This results in very high memory usage. For example using 8 parallel part uploads and uploading 280GB file I need to use 28MB parts (since S3 has limit of 10k parts). This results in 700MB of memory usage since we need 3 copies * 28 MB per part * 8 parallel uploads = 672 MB of Vec<u8> containing data.
In ideal world rusoto would take the Vec<u8> as value and return it back with request response for future reuse. Since current API is taking request via pointer this is not possible. I could not find the source of third copy of the buffer though - perhaps that can be easily avoided.

Hi, thanks for the report, I'll try to look into that this week!
I've been wondering for a while whether using Bytes instead of Vec<u8> might be better. It'd also help here since Bytes::clone only shadow-clones the reference counted smart pointer.
It also seems like you could recover the allocated buffer for re-use via Buffer::try_mut.
Do you think that would work for your use-case?

Yes, Bytes would work well I think. If I understand it correctly I would create BytesMut and read to it and then call .freeze() to get Bytes which I can .clone() to keep a copy before I pass it to your API. Once the upload is complete I would be able to call .try_mut().unwrap() on my copy and reuse resulting BytesMut for another read. All without extra allocations or copying.

Alternatively you could use Rc<Vec<u8>> or even Rc<Cursor<Vec<u8>> to avoid pulling in additional dependency.

In ideal world I would be giving something like AsyncRead or other Stream that would let me stream small bits of data for each part of S3 multipart upload and therefore reduce memory usage even further - currently I need to keep at least one part fully in memory and minimum size of it is dictated by S3 API restrictions (minimum size and max parts vs file size to upload).

Ah yeah, that seems like the more generic solution! We already have streaming support for responses, but that's still missing for requests.

@jpastuszek I have a PR in progress (https://github.com/rusoto/rusoto/pull/1030) to support streaming uploads for S3.
It has a few breaking changes unfortunately, but it'd be awesome if you could find the time to give that a shot and let me know how things go!

I have tested it with single buffer (using buf.into()) and I don't see 3x copies now. But looking at memory usage I still get quite substantial memory overhead: for 20MB Vec I see 12MB extra memory usage increase per parallel upload; for 40MB I see 27MB extra overhead per parallel upload.
CPU usage is no more stable - probably because we don't copy memory on start of every upload.
What I was expecting is to see when increasing parallelism (so parallel part uploads) is increase of memory dominated by size of the extra buffer + some constant for HttpClient etc..
Do you allocate memory somewhere proportional to stream length or buffer size?

As far as I can see, rusoto now does zero copies of the request bodies when streaming, but it's entirely possible that hyper does something internally. To clarify: When you say you see extra memory usage, do you mean a constant overhead that's relative to buffer size, or an overhead that's relative to total body size?

What I do is I start say 7 upload_part and give each single 20MB Vec (stream::once). If I start 8 uploads the difference in memory usage (Private Bytes on Windows) is ~32MB (so my 20MB buf + 12MB for something). Now if I start 7 uploads with 40MB Vecs the difference is ~64MB (so my 40MB buf + 24MB for something) vs 8 uploads. So it looks proportional to size of the Vec/part.
Now I will try to stream each part with smaller Vecs and see how this works out.

Great, let me know how you go!

I have figured out how to stream file parts in chunks.
It all works well on Mac with memory usage proportional to the size of buffers I use for streaming parts and parallelism and not depending on the size of the parts. I can do 8 parallel streamed part uploads with 256KiB double buffering for each stream in 17MB of Private Mem constant.
Now with same settings but on Windows I get 164MB of Private Bytes used. Memory usage depends on the size of the part. If I increase part size to 100MiB my client uses 1GB of RAM.
It looks like the streamed buffers are not deallocated until part upload is complete on Windows.
About the API itself the tricky bit is to get the flow control right. In my client I use constant number of buffers that circulate between file reader thread and streams I pass to StreamingBody. When I provide a buffer for upload via the stream I send one newly allocated buffer back to reader to keep the process going.
The problem with this API is that I have to give away buffers and keep allocating new buffers in their place. I guess this allocations are not a big problem performance wise but my ideal API would be consisting of stream that produces buffers for upload and a return stream that gives me back buffers after they have been sent so I can reuse them for further reads and get natural flow control for free.
Another issue is the len parameter in StreamingBody. I guess this the total size of the part or sum of lengths of the buffers I am going to provide. In this case it should probably be u64 and not usize since this limits size of single part while I can provide much more bytes with multiple buffers. Another thing is that it is mandatory - while with S3 uploads it will always be possible to calculate the size of the stream (part) upfront this might not be possible for other APIs where you can work with streams of unknown sizes. If this len is used to get Content-Length rather than chunked based framing then it should be optional and would also be duplicating UploadPartRequest.content_length parameter.

This is great feedback, thanks!
The memory usage on Windows sounds like a bug, but one that most likely lives upstream, in either hyper or tokio. It might be worth trying to reproduce the issue with a non-rusoto hyper client, and file a bug in the hyper project to follow this up.
Regarding allowing for re-use of buffers, I've used crates like pool for this in the past. Using vanilla hyper this would work, since hyper can be generic over buffer types, as long as they implement AsRef<u8>. With rusoto, it's a bit more tricky since we've currenly hard-coded the buffer type to Vec<u8>, and making it generic will have some pretty big knock-on effects as far as I can tell. We can explore this further if you want, and maybe we can integrate closer with something like pool to make this happen.
Lastly, thanks for pointing out the oversight with content_length. There was a small bug in the signing logic that prevented me from doing this earlier, but now I'm working on fixing it in #1043.

I have tried #1043 and it worked OK.
Now I am working on on-the-fly compression. This means I will not know the content length of the part upload unless I buffer it all up in memory.
When I set UploadPartRequest.content_length: None I get following error reply from S3:
 <?xml version="1.0" encoding="UTF-8"?> <Error>NotImplemented<Message>A header you provided implies functionality that is not implemented</Message>Transfer-Encoding<RequestId>A9D241AF04F6CB2A</RequestId><HostId>/Mwe9AfvE1fRHtMYqc7xoUYd4Rdt2bhyAkDTQzLmi0nXDsevITslhO/6OQ4LgbvYeTnEoL8RzzQ=</HostId></Error>

Do you know if S3 supports chunked encoding? Some sources say it requires special payload signing.

Closing this now that #1350 is merged! 🎉


documentation available ?

is there basic docs on how to use this ? I want to make a custom backend

Hi
Documentation is under progress
Meanwhile you can create an account at litekart.in for a lifetime free backend


Change deprecated exec_program to execute_process in iOS.cmake

See CMP0153
iOS builds fail. https://github.com/Swiftgram/TDLibFramework/actions/runs/7299002751/job/19891100843

Thank you!


-v :/var/www does not see files

Is there an issue with file permissions? When I try to pass a volume in the docker container I can no see the files in the volume.

@d1b1 Try https://jeremyfelt.com/2013/01/08/clear-nginx-cache-in-vagrant/?


UHF-8508 Select icon PoC

UHF-8508

What was done

Removed mentions of Select2 and Select2 Icon libraries.
Added Tom Select library.
Added field widgets, templates and theme hooks for color palette widget and design widget.
Rewrote all Select2 assets (js) to use Tom Select UI control. Added styles for the select lists.
Added - None - option for the select icon tool.
Handle image preview event listeners in a separate function.
Unified naming convention of select list components.
Fixed icon selection on multiple instances.
Prevent the image preview js to fire immediately after mouse move event.

How to install

Make sure your instance is up and running on latest dev branch.

git pull origin dev
make fresh

Update the necessary modules / themes

composer require drupal/helfi_platform_config:dev-UHF-8508_select_icon_poc drupal/hdbt_admin:dev-UHF-8508_select_icon_poc

Run make drush-updb drush-cr

How to test

Check that following features work

[x] Paragraph selection and paragraph preview image hover

[x] Design style selection

[x] Language selection in CKEditor

[x] Icon selection

[x] Icon selection in Link dialog in CKEditor

[x] Color palette selection in sidebar

Other PRs

https://github.com/City-of-Helsinki/drupal-hdbt-admin/pull/220

Closing as this is only a PoC. The codebase can be used as a base when proceeding with CKEditor 5 updates.

Reopened the PR as the PoC is a complete functionality and can be merged.


Noob Question: How To Use JVFloatLabeledTextField Using Storyboard

How Can I Use JVFloatLabeledTextField Using Storyboard??? I tried to drag an UITextField into view controller and set the class to JVFloatLabeledTextField class, but storyboard give me two error:
"Failed to update auto layout status: Failed to load designables from path (null)" and
"Failed to render instance of JVFloatLabeledTextField: Failed to load designables from path (null)"
Hope someone can help me regarding this problem. Thanks in advance. :)

After some experiment, I solved my problem by adding " use_frameworks! " in podfile and run pod update command in terminal. Visit this link for solution:
http://stackoverflow.com/questions/28204108/ib-designables-failed-to-update-auto-layout-status-failed-to-load-designables/28907253#28907253


github actions python version

Update abatilo/actions-poetry action to v3
Fix python version in github actions

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (56027bf) 52.05% compared to head (db565cf) 52.05%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #218   +/-   ##
=======================================
  Coverage   52.05%   52.05%
=======================================
  Files           2        2
  Lines          73       73
=======================================
  Hits           38       38
  Misses         35       35

Flag
Coverage Δ

unittests
52.05% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Set correct tile sizes on root ops with pack/unpack ops

Set correct tile sizes on root ops instead of using the same tile sizes as the following pack ops.
Currently to set the tile sizes for pack ops, we first use adjustTileSizesForPackOp to scale down the tile sizes on the root op according the the inner tile sizes then call setLoweringConfigForComputeOps to copy the scaled tile size to the pack op.
This results in misleading lowering config on the root op because its actual tile sizes is scaled up from the pack op later in the tile-and-fuse.
So instead of scaling down the tile sizes on root op, this change aligns the tile sizes on the root op to inner tile size, then scale down the sizes when we copy them to the pack op. This way it keeps lower configs on both root and pack op accurate.
Here is an example after this change. We can see that the lowering config on the generic op is set to [[64, 64], [16, 16]] (previous it would be [4, 64], [1, 16]). This correctly reflects the actual tiling size when it is tiled-and-fused with the pack op.
Input:
// RUN: iree-opt --pass-pipeline='builtin.module(hal.executable(hal.executable.variant(iree-llvmcpu-lower-executable-target{test-lowering-configuration=true})))' %s

hal.executable private @aligned_generic_pack {
  hal.executable.variant public @embedded_elf_x86_64, target = <"llvm-cpu", "embedded-elf-x86_64", {cpu_features = "+avx512f", data_layout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128", native_vector_size = 64 : index, target_triple = "x86_64-none-elf"}> {
    hal.executable.export public @aligned_generic_pack ordinal(0) layout(#hal.pipeline.layout<push_constants = 0, sets = [<0, bindings = [<0, storage_buffer, ReadOnly>, <1, storage_buffer>]>]>) {
    ^bb0(%arg0: !hal.device):
      %c1 = arith.constant 1 : index
      hal.return %c1, %c1, %c1 : index, index, index
    }
    builtin.module {
      func.func @aligned_generic_pack() {
        %c0 = arith.constant 0 : index
        %cst = arith.constant 3.40282347E+38 : f32
        %cst_0 = arith.constant 0.000000e+00 : f32
        %0 = hal.interface.binding.subspan set(0) binding(0) type(storage_buffer) alignment(64) offset(%c0) flags(ReadOnly) : !flow.dispatch.tensor<readonly:tensor<512xf32>>
        %1 = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) alignment(64) offset(%c0) flags(ReadOnly) : !flow.dispatch.tensor<readonly:tensor<384x512xf32>>
        %2 = hal.interface.binding.subspan set(0) binding(2) type(storage_buffer) alignment(64) offset(%c0) : !flow.dispatch.tensor<writeonly:tensor<24x512x16x1xf32>>
        %3 = flow.dispatch.tensor.load %0, offsets = [0], sizes = [512], strides = [1] : !flow.dispatch.tensor<readonly:tensor<512xf32>> -> tensor<512xf32>
        %4 = flow.dispatch.tensor.load %1, offsets = [0, 0], sizes = [384, 512], strides = [1, 1] : !flow.dispatch.tensor<readonly:tensor<384x512xf32>> -> tensor<384x512xf32>
        %5 = tensor.empty() : tensor<24x512x16x1xf32>
        %6 = tensor.empty() : tensor<384x512xf32>
        %7 = linalg.generic {indexing_maps = [affine_map<(d0, d1) -> (d1)>, affine_map<(d0, d1) -> (d0, d1)>, affine_map<(d0, d1) -> (d0, d1)>], iterator_types = ["parallel", "parallel"]} ins(%3, %4 : tensor<512xf32>, tensor<384x512xf32>) outs(%6 : tensor<384x512xf32>) {
        ^bb0(%in: f32, %in_1: f32, %out: f32):
          %8 = arith.addf %in, %in_1 : f32
          %9 = arith.minimumf %8, %cst : f32
          %10 = arith.maximumf %9, %cst_0 : f32
          linalg.yield %10 : f32
        } -> tensor<384x512xf32>
        %pack = tensor.pack %7 inner_dims_pos = [0, 1] inner_tiles = [16, 1] into %5 : tensor<384x512xf32> -> tensor<24x512x16x1xf32>
        flow.dispatch.tensor.store %pack, %2, offsets = [0, 0, 0, 0], sizes = [24, 512, 16, 1], strides = [1, 1, 1, 1] : tensor<24x512x16x1xf32> -> !flow.dispatch.tensor<writeonly:tensor<24x512x16x1xf32>>
        return
      }
    }
  }
}

Output:
#config = #iree_codegen.lowering_config<tile_sizes = [[64, 64], [16, 16], [0, 0], [0, 0]]>
#config1 = #iree_codegen.lowering_config<tile_sizes = [[4, 64], [1, 16], [0, 0], [1, 16]]>
#executable_target_embedded_elf_x86_64_ = #hal.executable.target<"llvm-cpu", "embedded-elf-x86_64", {cpu_features = "+avx512f", data_layout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128", native_vector_size = 64 : index, target_triple = "x86_64-none-elf"}>
#map = affine_map<(d0, d1) -> (d1)>
#map1 = affine_map<(d0, d1) -> (d0, d1)>
#pipeline_layout = #hal.pipeline.layout<push_constants = 0, sets = [<0, bindings = [<0, storage_buffer, ReadOnly>, <1, storage_buffer>]>]>
#translation = #iree_codegen.translation_info<CPUDoubleTilingExpert>
module {
  hal.executable private @aligned_generic_pack {
    hal.executable.variant public @embedded_elf_x86_64, target = #executable_target_embedded_elf_x86_64_ {
      hal.executable.export public @aligned_generic_pack ordinal(0) layout(#pipeline_layout) attributes {translation_info = #translation} {
      ^bb0(%arg0: !hal.device):
        %c1 = arith.constant 1 : index
        hal.return %c1, %c1, %c1 : index, index, index
      }
      builtin.module {
        func.func @aligned_generic_pack() {
          %c0 = arith.constant 0 : index
          %cst = arith.constant 3.40282347E+38 : f32
          %cst_0 = arith.constant 0.000000e+00 : f32
          %0 = hal.interface.binding.subspan set(0) binding(0) type(storage_buffer) alignment(64) offset(%c0) flags(ReadOnly) : !flow.dispatch.tensor<readonly:tensor<512xf32>>
          %1 = hal.interface.binding.subspan set(0) binding(1) type(storage_buffer) alignment(64) offset(%c0) flags(ReadOnly) : !flow.dispatch.tensor<readonly:tensor<384x512xf32>>
          %2 = hal.interface.binding.subspan set(0) binding(2) type(storage_buffer) alignment(64) offset(%c0) : !flow.dispatch.tensor<writeonly:tensor<24x512x16x1xf32>>
          %3 = flow.dispatch.tensor.load %0, offsets = [0], sizes = [512], strides = [1] : !flow.dispatch.tensor<readonly:tensor<512xf32>> -> tensor<512xf32>
          %4 = flow.dispatch.tensor.load %1, offsets = [0, 0], sizes = [384, 512], strides = [1, 1] : !flow.dispatch.tensor<readonly:tensor<384x512xf32>> -> tensor<384x512xf32>
          %5 = tensor.empty() : tensor<24x512x16x1xf32>
          %6 = tensor.empty() : tensor<384x512xf32>
          %7 = linalg.generic {indexing_maps = [#map, #map1, #map1], iterator_types = ["parallel", "parallel"]} ins(%3, %4 : tensor<512xf32>, tensor<384x512xf32>) outs(%6 : tensor<384x512xf32>) attrs =  {lowering_config = #config} {
          ^bb0(%in: f32, %in_1: f32, %out: f32):
            %8 = arith.addf %in, %in_1 : f32
            %9 = arith.minimumf %8, %cst : f32
            %10 = arith.maximumf %9, %cst_0 : f32
            linalg.yield %10 : f32
          } -> tensor<384x512xf32>
          %pack = tensor.pack %7 inner_dims_pos = [0, 1] inner_tiles = [16, 1] into %5 {lowering_config = #config1} : tensor<384x512xf32> -> tensor<24x512x16x1xf32>
          flow.dispatch.tensor.store %pack, %2, offsets = [0, 0, 0, 0], sizes = [24, 512, 16, 1], strides = [1, 1, 1, 1] : tensor<24x512x16x1xf32> -> !flow.dispatch.tensor<writeonly:tensor<24x512x16x1xf32>>
          return
        }
      }
    }
  }
}

Fix #14940

Dispatch sizes are changing so this means the fix is actually changing the generated code. I wonder if we have a place where we pick up the incorrect tile sizes of the generic op and check if there is a pack op around and then scale the tile sizes based on the pack op info...

Sorry I didn't pay attention because there was no clear regressions on latencies.
I checked locally and seems like the VMFBs are identical before and after this change. I suspect the dispatch size changes here are due to the outdated baseline (https://github.com/openxla/iree/commit/fb9e1b68f73ace6ae799ed27ad06a1288deccdb6) we got from the perf dashboard, which is before an LLVM integration commit.

Sorry I didn't pay attention because there was no clear regressions on latencies.
I checked locally and seems like the VMFBs are identical before and after this change. I suspect the dispatch size changes here are due to the outdated baseline (fb9e1b6) we got from the perf dashboard, which is before an LLVM integration commit.

Confirmed it was the integration changed the dispatch sizes (its results hadn't been submitted at that time, however Github CI always rebases and runs PR on the ToT)
https://perf.iree.dev/serie?IREE?56bc9128e294585d749b8ebe34fd03060ba34d200eef185837b6002d0dcbfccb-9e15f7e6-383c-47ec-bd38-ecba55a5f10a

Awesome! Thanks for confirming


merging integrations into main test branch

[Current coverage][cc-pull] is 92.05%

No coverage report found for tdd at b20e589.

Powered by Codecov. Last updated by [b20e589...a4179ed][cc-compare]
[cc-base-branch]: https://codecov.io/gh/quinkennedy/spacebrew/branch/tdd?src=pr
[cc-compare]: https://codecov.io/gh/quinkennedy/spacebrew/compare/b20e589d9205484589cdff79efe7cd0bede52072...a4179ed4447baaef4db245c1827cbad705c37459?src=pr
[cc-pull]: https://codecov.io/gh/quinkennedy/spacebrew/pull/3?src=pr

Changes Unknown when pulling 815eceae1596b2c77925ad184170df0f2efd476f on integrations into ** on tdd**.

Changes Unknown when pulling 66ce41ee779246e6e39c2292d72c35d838d15e33 on integrations into ** on tdd**.

Changes Unknown when pulling 1bdd013c9a969d87901a5fa73caaee7131b4049e on integrations into ** on tdd**.


redundant links in the navigation menu

The "How Artists Get Paid from Streaming" story, and likely some others, includes the below navigation menu. As a result of last year's site revamp, the first three bulleted links all lead you to the same (home)page. Could we lose the first two?

this is actually related to the starter template, so I'm closing this here

fixed starter template codebase


texlive: fix latexindent runtime

latexindent currently fails due to missing perlPackages dependencies
Resolves: #54452
This was tested using
nix-shell -p "texlive.combine { inherit (texlive) scheme-minimal latexindent; }" \
  --pure --run latexindent

lib.closePropagation is deprecated, so we shouldn't use it, probably. Normally the closure would be constructed by adding packages to buildInputs and dumping resulting $PERL5LIB into the wrapper. This change will become much more obtrusive if I go that way.

This is no good. Should use buildPerlPackage instead.


Fibaro hub component not loading

Home Assistant release with the issue:

0.83.0
Last working Home Assistant release (if known):
New component
Operating environment (Hass.io/Docker/Windows/etc.):

hass.io on raspberry pi3
Component/platform:

https://www.home-assistant.io/components/fibaro/
Description of problem:
Component not loading on startup.
Please note that I have a "big" setup in fibaro with 30+ components in HC2 and 2 slave gateways (HCL). Perhaps the error is related to this ?
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

fibaro:
  url: http://<IP_ADDRESS>/api/
  username: !secret fibaro_user
  password: !secret fibaro_password

Traceback (if applicable):
2018-11-29 21:01:15 ERROR (MainThread) [homeassistant.setup] Error during setup of component fibaro
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/homeassistant/setup.py", line 148, in _async_setup_component
    component.setup, hass, processed_config)  # type: ignore
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/site-packages/homeassistant/components/fibaro.py", line 203, in setup
    if controller.connect():
  File "/usr/local/lib/python3.6/site-packages/homeassistant/components/fibaro.py", line 89, in connect
    self._read_devices()
  File "/usr/local/lib/python3.6/site-packages/homeassistant/components/fibaro.py", line 180, in _read_devices
    (not device.isPlugin or self._import_plugins):
  File "/usr/local/lib/python3.6/site-packages/fiblary3/client/v4/models.py", line 120, in __getitem__
    return dict.__getitem__(self, key)
KeyError: 'isPlugin'

Additional information:
I hope I filled in everything right, first time I report a bug
It was fantastic news that fibaro component was added, before I have had to get each sensor with my own modified REST API component

@pbalogh77

@jengstr could you please try with 0.83.1 or 0.83.2 release? It was fixed in 0.83.1, having virtual devices crashed the component on startup, it should be working fine now.

Solved in 0.83.1.
Thank you very much (now I am only missing the support for multiple fibaro hubs... but I added that as a separate request.)


BCD: border-block-end

https://developer.mozilla.org/en-US/docs/Web/CSS/border-block-end is supported in Chrome. I assume all the associated border-block properties are too. We need to check all and update the BCD

I've been going round these recently so I can do these ones as well.

PR for this data: https://github.com/mdn/browser-compat-data/pull/3717

PR is merged so I think this is all done.