Only test via public APIs, and don't mock or stub SUT

I started investigative work on extracting adapter gems, and almost immediately ran into issues with broken tests, because the tests were tightly coupled with the implementation details. These issues generally surfaced in two forms: testing private methods instead of testing behavior via the public API, mocking and stubbing internal methods directly on the system under test instead of the system's collaborators.
This PR addresses those issues, opening the door for friction-less refactoring and feature addition. Hopefully this is the last yak-shave before we can start delivering real value i.e. adapter gems!

@etagwerker Okay, just resolved the merge conflict, so this should be ready to go! :green_heart:
After this is merged, I can push my gem extraction branch up for your review. It's happening! :tada:


optimize UI for call centers

This mainly means ensuring the UI either by default or by a user preference can be made optimized for keyboard use. I am thinking here especially about navigation and search.
Semi related this to me also means supporting less structured search. I know there is the global search UI which is a good step. I am not sure if such a "search across all fields" search widget is available for the datagrid. Also of course supporting Lucene based search engines may be necessary here too (I know that this is on the roadmap).

@lsmith77  have you tested https://marketplace.orocrm.com/package/oro-hotkeys ?
Related to https://github.com/orocrm/platform/issues/432


Error while running make.sh

I am getting the following error message while running make.sh in the models/ops directory.
I am exactly following the installation steps provided in the README file
`Traceback (most recent call last):
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1423, in _run_ninja_build
check=True)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 70, in
cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 603, in build_extensions
build_ext.build_extensions(self)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
_build_ext.build_extension(self, ext)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
depends=ext.depends)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 437, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1163, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/home/aditya_rastogi/anaconda3/envs/ddetr/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1436, in _run_ninja_build
raise RuntimeError(message)
RuntimeError: Error compiling objects for extension`

Have you solved this problem?

Have you solved this problem?

I meet the same problem

I have the same problem with cuda11.0、python3.7 and torch1.7.0, and have you solved this issues?


Copy on select with separate buffers

Could we have the same copy on select as in Ubuntu. The feature should hopefully be in  windows os so that you could copy from web browser into terminal.

Copy/paste is not a function of WSL. It is a feature of Windows terminal. [The topic is pretty well trodden over there so do a search before opening a new issue.]


Changed the GIF for cloning this repo

Please remove "7.15 MB/s" after the line ending with "total packet reused..."

Okay Sure!


require is not defined when putting ipcRenderer.on in /src/renderer.ts

Hi!!
Thank you for this excellent electron-react-webpack-typescript starting point.
I'm trying to start from it to transpose within it this other simple example of two windows communication:https://github.com/kahlil/electron-communication-example
But I'm encountering this hurdle: when inserting into /src/renderer.ts these lines:
ipcRenderer.on('messageFromMain', (event, message) => {
  console.log(`This is the message from the second window sent via main: ${message}`);
});

I get this error:

This is the entire /src/renderer.ts file:
import './app';

console.log('renderer.js is so happy to say you 👋 hello....');

import { ipcRenderer, BrowserWindow } from 'electron';

ipcRenderer.on('messageFromMain', (event, message) => {
  console.log(`This is the message from the second window sent via main: ${message}`);
});

Based on what I found here: // https://stackoverflow.com/questions/44008674/how-to-import-the-electron-ipcrenderer-in-a-react-webpack-2-setup , I've also put in tools/webpack/webpack.plugins.js :
const webpack = require('webpack');

module.exports = [
  new ForkTsCheckerWebpackPlugin(),
 new webpack.ExternalsPlugin('commonjs', [
    'electron'
  ])
];

I tried to import electron in renderer.ts as a window component:
const electron = window.require('electron');

But got:
window.require is not a function

It must be something related to the interaction among electron, react and webpack, but I didn' find any solution yet.
Would you be so kind in helping me?

Hello @raphael10-collab
Its good to know this boilerplate been a good starting point for you.
For the issues you mentioned regarding:
require is not defined

Can you try the same process with following WebPreference configuration in main.ts file for Electron Window
(similar to https://github.com/codesbiome/electron-react-webpack-typescript-2020/issues/4#issuecomment-728118263)
    webPreferences: {
      nodeIntegration: true, // Changed
      enableRemoteModule: false,
      contextIsolation: false, // Changed
      nodeIntegrationInWorker: false,
      nodeIntegrationInSubFrames: false,
      preload: MAIN_WINDOW_PRELOAD_WEBPACK_ENTRY,
    },

In this boilerplate nodeIntegration & contextIsolation settings are set to false by default for security purposes

With
webPreferences: {
  nodeIntegration: true, // Changed
  enableRemoteModule: false,
  contextIsolation: false, // Changed
  nodeIntegrationInWorker: false,
  nodeIntegrationInSubFrames: false,
  preload: MAIN_WINDOW_PRELOAD_WEBPACK_ENTRY,
},

it doesn't give
require is not defined
when inserting in renderer.ts:
ipcRenderer.on('messageFromMain', (event, message) => {
  console.log(`This is the message from the second window sent via main: ${message}`);
});

So.. the hurdle now changes:
how to keep for security purposes
nodeIntegration:false,
contextIsolation: true,

and being able in the same time to import ipcRenderer in renderer.ts?
There must be a wayto make renderer.ts and main.ts communicate with each other, while keeping the environment as much as secure as possibile.

How about using/importing ipcRender into preload.ts instead of  renderer.ts ?

There must be a way to make renderer.ts and main.ts communicate with each other, while keeping the environment as much as secure as possibile.

For the communication between both main and renderer process, how about using following methods instead:
// Main process
ipcMain.handle('my-invokable-ipc', async (event, ...args) => {
  const result = await somePromise(...args)
  return result
})

// Renderer process
async () => {
  const result = await ipcRenderer.invoke('my-invokable-ipc', arg1, arg2)
  // ...
}

When importing ipcRenderer in preload.ts instead of renderer.ts:
import { ipcRenderer } from 'electron'; // in preload.ts

It seems that renderer.ts doesn't recognize ipcRenderer :
ERROR in src/renderer.ts:14:24
TS2304: Cannot find name 'ipcRenderer'.
    12 |
    13 | async () => {
  > 14 |   const result = await ipcRenderer.invoke('my-invokable-ipc', "this is my message")
       |                        ^^^^^^^^^^^
    15 |   // ...
    16 | }

I defined the 'electron-renderer' as target in /tools/webpack/webpack.renderer.js :
/* eslint-disable @typescript-eslint/no-var-requires */
const rules = require('./webpack.rules');
const plugins = require('./webpack.plugins');
const aliases = require('./webpack.aliases');

module.exports = {
  target: 'electron-renderer',
  module: {
    rules,
  },
  plugins: plugins,
  resolve: {
    extensions: ['.js', '.ts', '.jsx', '.tsx', '.css'],
    alias: {
      // React Hot Loader Patch
      'react-dom': '@hot-loader/react-dom',
      // Custom Aliases
      ...aliases,
    },
  },
};

But still get the error :
ERROR in src/renderer.ts:14:24
TS2304: Cannot find name 'ipcRenderer'.

Just tried using following in preload.ts it shows console output without any issue:
// preload.ts

import { ipcRenderer } from 'electron';
console.log(ipcRenderer);

I tried to import ipcRenderer in preload.js as follows:
window.ipcRenderer = require('electron').ipcRenderer;

Instead of exposing ipcRenderer to the window (global) object, try making use of context-bridge approach within preload.ts to expose custom APIs for application (browser) window. https://www.electronjs.org/docs/api/context-bridge

I just followed the example in https://www.electronjs.org/docs/api/context-bridge#contextbridge
preload.js :
import { ipcRenderer, contextBridge } from 'electron';
contextBridge.exposeInMainWorld(
  'electron',
  {
    doThing: () => ipcRenderer.send('do-a-thing')
  }
)

renderer.js  :
import './app';

console.log('renderer.js is so happy to say you 👋 hello....');

window.electron.doThing();

Error:
ERROR in src/renderer.ts:5:8
TS2551: Property 'electron' does not exist on type 'Window & typeof globalThis'. Did you mean 'Electron'?
    3 | console.log('renderer.js is so happy to say you 👋 hello....');
    4 |
  > 5 | window.electron.doThing();
      |        ^^^^^^^^

ERROR in src/renderer.ts:5:8
TS2551: Property 'electron' does not exist on type 'Window & typeof globalThis'. Did you mean 'Electron'?
    3 | console.log('renderer.js is so happy to say you 👋 hello....');
    4 |
  > 5 | window.electron.doThing();
      |        ^^^^^^^^

Following is compilation error by typescript compiler because of window.electron object unknown property type.
You will need to declare these additional properties as custom type/interface for Window object inside index.d.ts file, for example :
// index.d.ts

declare interface Window {
  electron: {
    doThing(): void
  }
}

Yes. Now it seems working with:
in main.js
webPreferences: {
  nodeIntegration: false,
  enableRemoteModule: false,
  contextIsolation: true,
  nodeIntegrationInWorker: false,
  nodeIntegrationInSubFrames: false,
  preload: MAIN_WINDOW_PRELOAD_WEBPACK_ENTRY,
},

preload.js :
import { ipcRenderer, contextBridge } from 'electron';

contextBridge.exposeInMainWorld(
  'electron',
  {
    doThing: () => ipcRenderer.send('do-a-thing')
  }
)

index.d.ts :
declare interface Window {
  electron: {
    doThing(): void
  }
}

and renderer.ts :
import './app';

console.log('renderer.js is so happy to say you 👋 hello....');

window.electron.doThing();

Thank you very much @codesbiome  !!!!

Your welcome @raphael10-collab
Cheers!
This issue will be converted into Discussion.


EndToEnd tests failing(hanging) on latest 4.0 snapshots on Linux

With the last few swift-4.0 development snapshots, all but one of the XCTests with names that end with EndToEnd fail on Linux with timeouts.  These are the tests that start a server on a random port and create a URLSession to send requests to it  Those tests pass fine on Darwin (and on Linux under Swift 3.1.1).
If you comment all but one of the EndToEnd tests out, then the one you leave in (which ever one you pick) will pass.  With two EndToEnd tests in, regardless of the order you choose, the first one will complete, and the second one will time out.
I'm guessing something isn't being cleaned up correctly, but I'm not sure what. Will update as I learn more.

I updated BlueSockets to the latest 4.0 toolchain (see https://github.com/IBM-Swift/BlueSocket/tree/UnsupportedWIP/) to see if that was the problem, and the BlueSocket tests run clean on Linux under the latest snapshots, so that's not it.
The investigation continues.

It appears that the issue is happening because, in the test that passes, the attempt to close the socket fails (although the test passes).
Here's the working output (with some extra prints) on the Mac:
Test testHelloEndToEnd() on port 53808
Connection Accepted
Listener called
process setting blocking mode
process resuming reader source <OS_dispatch_source: 0x7f8e8cf0b170> on socket 13
reader source event
reader source event about to read
reader source read 223 bytes
reader source sending to parser
reader source returned from parser
reader source event about to read
close called
socket valid calling close on it
checking reader queue <OS_dispatch_queue_serial: Read Queue 0[0x7f8e8ce2ef40]> before calling cancel
ReaderSource Event Error: Error code: -9982(0x-26FE), Bad file descriptor
close called
checking reader queue <OS_dispatch_queue_serial: Read Queue 0[0x7f8e8ce2ef40]> before calling cancel
reader source cancel
Calling close() on socket -1
close called

And here's the same (failing) call on Linux:
Test testOkEndToEnd() on port 43869
Connection Accepted
Listener called
process setting blocking mode
process resuming reader source Dispatch.DispatchSource on socket 9
reader source event
reader source event about to read
reader source read 228 bytes
reader source sending to parser
reader source returned from parser
close called
socket valid calling close on it
checking reader queue Dispatch.DispatchQueue before calling cancel
checking reader source Dispatch.DispatchSource before calling cancel
Test Case 'ServerTests.testOkEndToEnd' passed (0.021 seconds)

The two diverge at: ReaderSource Event Error: Error code: -9982(0x-26FE), Bad file descriptor
On the Mac when this happens, it indicates that the socket was closed as it was being read.  This error doesn't happen on Linux (although the same close() call does), leaving the socket open so that the next (and subsequent) tests hang.
Next step to see if I'm correct, is to replicate this error in a much simpler sample project.

Here is the output of two test runs on the same system, one from Swift3.1.1 and the other from last week's 4.0 Snapshot:
swift.test.311.txt
swift.test.40.txt
In both cases, the code is doing a blocking call to Glibc.recv() in a DispatchSource's Read event handler when Glibc.shutdown() is called.  In Swift 3.1.1, we get Got Count 0 from Glibc.recv(-1), but in Swift 4.0, the Glibc.recv() call doesn't return.

Possibilities:

It's possible the system calls are behaving differently, either:

The system calls do something different when being called from Swift4 than when called from Swift3. I'd rate this unlikely.
There's some global state that's set differently in Swift4 than Swift3 that's affecting the system calls. Possible, but I can't think of an obvious place to start looking.
Some other reason.

The system calls are behaving the same, but printing is broken, rendering my data invalid.
The system calls are behaving the same but some kind of threading/concurrency problem is keeping things from being able to continue, causing the thread/queue calling recv() to hang.
Something else.

Hi Carl.  Am I reading it correctly that this test used to work on Swift 4 development snapshots until about a week ago?    Asking because Apple merged several of their own PRs into dispatch during the July 13-July 17 time period.  I'm trying to figure out if it is worth selectively backing those out to see if one of them was buggy.  They touched bits of code dealing with Linux signal handling, ioctls, and processing of epoll.

As far as i know, it's never worked on Swift 4. It certainly hasn't worked on any of the 4.0-DEVELOPMENT snapshots that I've tried. It has worked on all the 3.x ones.
One of the things I will do if I run out of other things to try is to go back in time snapshot by snapshot until I find the one where it stopped working.  That will be a lot of downloading snapshots, though, so I was hoping to figure it out some other way.

ok, thanks.  Then I will look from first principles instead of focusing on what Apple changed in their PRs last week.

Ok, so good news and bad news.
The good news is, I've discovered via brute force that the regression happens between the 2017-05-30 and 2017-05-31 snapshots (e.g. the tests pass with the May 30th snapshot and hang/fail with the May 31st one).
The bad news is there are no libDispatch changes between the swift-DEVELOPMENT-SNAPSHOT-05-30-a and swift-DEVELOPMENT-SNAPSHOT-05-31-a tags. So if the snapshots are representative of the tags, it's not a libDispatch issue.
The worse news is that of the changes between those tags, the biggest change is the addition of URLProtocol.

sounds like its over to the Foundation team then.

Quite possibly. I'm just really not sure how a change to URLSession (happening on one queue) could possibly cause Glibc.close(socket) not to interrupt Glibc.recv(Socket) on a different queue.

@pushkarnk

I've discovered can fix this problem by applying a change to BlueSocket where we call Glibc.shutdown() when close()ing a socket that isConnected in addition to isListening. (c.f. https://github.com/carlbrown/BlueSocket/commit/f5dc916bf707b12b11634a89655f92c3836f5051#diff-cec92f2c4685d05c3d246f2bb8a42733L3324).  This means that BlueSocket will call shutdown() on both the TCP connection being used to transfer data and the TCP socket that is listening for new connections (instead of just closing the listening socket and hoping that cleans up the accepted ones as well).
I think this is probably the right thing to do in any case (well, really I think BlueSocket should have exposed a shutdown() method, but that ship has sailed).  However, that raises the problem that we have no way of knowing what other repositories out in the wild might break because they are relying on the existing behavior.
The other problem is: that "fix" in no way answers the question about why the previous version works fine on the Mac and on pre-May31st/pre-_HTTPURLProtocol versions of Swift, but fails on post-May30th/post-_HTTPURLProtocol versions.
So I'm not sure if I should apply the BlueSocket fix and just move on, or keep trying to answer the question about what changed, expecting that this won't be the only place that the post-_HTTPURLProtocol behavior might be causing people issues once they move to Swift4.
Opinions? @ianpartridge ? @pushkarnk ?

I tried to get the tests running, but the swift test command fails for me with this:
$ ../swift/usr/bin/swift test
Linking ./.build/x86_64-unknown-linux/debug/SwiftServerHTTPPackageTests.xctest
/usr/bin/ld.gold: error: cannot find -lCHTTPParser
/usr/bin/ld.gold: error: cannot find -lCHTTPParser
/usr/bin/ld.gold: error: cannot find -lCHTTPParser
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
error: terminated(1): /root/pushkar/swift/usr/bin/swift-build-tool -f /root/pushkar/http/.build/debug.yaml test

Any idea what I am missing here?

You have to remove the link line from the module.map for CHTTPParser.

There's also a tar file in a comment above (https://github.com/swift-server/http/issues/22#issuecomment-316462312) that has the whole thing, if that's easier.

Thanks @carlbrown I can see the failures now. I'll start the investigation.

Any status on this @pushkarnk @ianpartridge ?

@carlbrown I see that before the failing tests this message is printed:
Error accepting client connection: Error code: -9994(0x-270A), Invalid argument
Does it mean the test isn't able to connect to the server? Do you see this message in your env?

@pushkarnk No, that means the socket was shutdown while the accept() system call was listening for new connections (which invalidates the socket passed as an argument).
You'll see that message on both the May 30th and May 31st snapshots.

The 'ReaderSource Event Error: Error code: -9982(0x-26FE), Bad file descriptor' message comes on Linux, when running the tests in LLDB and the tests seem to always pass.

What snapshot are you seeing them passing with? What changed since 29 days ago when you said you saw the failures?

@carlbrown I don't see them pass with simply run as swift test. They pass when the binary is run in LLDB. With GDB I see the hang. This is with the latest toolchain. I am trying to understand how a debugger can make a difference.

fyi: I have more detailed updates of my investigation on this issue
https://github.com/IBM-Swift/SwiftRuntime/issues/428

https://github.com/apple/swift-corelibs-foundation/pull/1195 has fixed this problem.
URLSession holds references to URLSessionTask. URLSessionTask holds a reference to an EasyHandle, which is a wrapper around a curl easy handle. Per this doc, to close a connection pertaining to an easy handle, curl_easy_cleanup needs to be called. Now, we call curl_easy_cleanup in the deinit for EasyHandle. In the early days of URLSession, to keep URLSessionTasks alive beyond the function in which they were locally held (because, the responses are async), we had introduced a retain cycle between URLSession and URLSessionTask, and we broke it after the task completed. Unfortunately, this cycle breaking was missed in the refactoring exercise. As a result, URLSessionTask objects accumulated in memory, holding EasyHandles along with them. EasyHandle.deinit() and hence curl_easy_cleanup were never invoked, and hence connections were kept alive. Hence this problem in swift-server/http. The client end did not initiate a connection shutdown and the server was stuck in read() blocking the queue on which reads() for the subsequent connections waited indefinitely.
swift test now goes through with 0 failures.


Experimental feature support for TFLite selective build

Click to expand!
Issue Type
Feature Request
Have you reproduced the bug with TF nightly?
Yes
Source
source
Tensorflow Version
2.11-tflite-android
Custom Code
No
Current Behaviour?
I have followed the tutorial and made my TextCNN model have the ability of on-device-training. It works well with the prebuilt library for tensorflow-lite.
When trying to reduce the binary size
with the docker environment I found that the Java API such as Interpreter#runSignature is a part of experimental feature, which not enabled by the tensorflow/lite/tools/build_aar.sh.
I have tried to add experimental = True to tflite_custom_android_library() function on line 73 of the build_aar.sh and rebuild, got the tensorflow-lite.aar and tensorflow-lite-select-tf-ops.aar.
It seems that tensorflow-lite.aar does have the experimental feature code and works. But tensorflow-lite-select-tf-ops.aar failure to run on Android device with error:
java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZNK6google8protobuf7Message11GetTypeNameEv" referenced by "/data/app/*/lib/arm64/libtensorflowlite_flex_jni.so"...

I could not find any flag like experimental = True for select-tf-ops to enable. Is it possible to enable experimental features for TFLite selective build?
Relevant log output
No response

The problem has been solved.
https://github.com/tensorflow/tensorflow/issues/59102 add the option echo -n "build --config=monolithic" >> /tensorflow_src/.bazelrc can avoid the _ZNK6google8protobuf7Message11GetTypeNameEv error.


Naming and refactoring of Orchestrator and UI components

Renames:

Orchestrator (new base class)

Assistant -> Resolver: First implementation of an Orchestrator

PlanningAssistant -> Planner: Second implementation of an Orchestrator

UI (new base class for high-level UI)

LumenAI -> ChatUI

LumenAIExplorer -> ExplorerUI

Also I'm still in the process of factoring the Assistant controls out and re-adding them at the UI level.

My gut is leaning towards Coordinator over Orchestrator but no strong feelings. Resolver wasn't intuitive to me when I first saw it until I read the docstring--also no strong feelings.
All the other names makes sense to me.

Sounds good, happy with Coordinator instead. I think I'll probably stick with Planner but you're right that Resolver isn't great.

Planning to merge this and start the prompt consolidation refactor soon. We can update the resolver name later if we can think of a better name.

What about Solver? Or DependencySolver or DependencyResolver instead of Resolver

Merging to start refactoring the prompts; we can change the names later if we decide on something.


Speed up reflex CLI imports

Improves the speed of reflex CLI commands with lazy imports.
Before:
~/code/reflex main ❯ hyperfine "poetry run reflex -v"
Benchmark 1: poetry run reflex -v
  Time (mean ± σ):     860.1 ms ± 268.8 ms    [User: 574.2 ms, System: 132.2 ms]
  Range (min … max):   746.4 ms … 1623.6 ms    10 runs

After:
~/code/reflex nikhil/lazy2 ❯ hyperfine "poetry run reflex -v"
Benchmark 1: poetry run reflex -v
  Time (mean ± σ):     478.0 ms ±  17.0 ms    [User: 320.9 ms, System: 86.3 ms]
  Range (min … max):   460.2 ms … 516.4 ms    10 runs

Tested deploying with the PR, some commands are visibly faster than before. Very nice.


:lipstick: update footer styles

Updated the footer social icons styling using react icons

issue #58

also apply the hacktoberfest-accepted tag to the pr


Non-hex colors no longer work

I have a custom colorscheme that uses transparent as its base00 color, but after ca3ee8343118c53b8bad02d96466be2417f1f855 this no longer works since it now requires the color to be in hexadecimal. Are non-hexadecimal colors no longer supported or is this considered a bug?

I put in a guard for this use case, but tbh it's kind of defeating the purpose of using base16 colorscheme when you have 15 colours.


Can't get the dim of input when the Dataset is from generator.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 1.6
Python version: 3.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:
Exact command to reproduce:

Describe the problem
When use from_generator way to get dataset, it can't get the output shape of it. And then pass the features of it to the layers, the layer can't get the rank of it.
But it is wok well use from_tensor_slices way to get the dataset.
I am not sure if it is a bug. The tutorial mainly use from_tensor_slices as emamples. But I think the action should be same when use these ways to get the dataset.
Source code / logs
    def csv_yield(csv_path):
        with open(csv_path, newline='') as f:
            next(f)  # skip the first line
            reader = csv.reader(f)
            features, labels = [], []
            for line in reader:
                feature, label = [float(i) for i in line[:-1]], int(line[-1])
                yield feature, label

    ds_train = tf.data.Dataset.from_generator(lambda: csv_yield(r"../data/iris/iris_training.csv"),
                                              output_types=(tf.float32, tf.int32))
    ds_train = ds_train.shuffle(1000).batch(16).repeat(5)
    features, labels = ds_train.make_one_shot_iterator().get_next()

    net = tf.layers.dense(features, units=10, activation=tf.nn.relu)

ValueError: Input 0 of layer dense_1 is incompatible with the layer: its rank is undefined, but the layer requires a defined rank.
But is is ok when use tf.data.Dataset.from_tensor_slices, The only difference between of them is the way of dataset. They shouldn't have different actions.
    def read_csv(csv_path):
        with open(csv_path, newline='') as f:
            next(f)  # skip the first line
            reader = csv.reader(f)
            features, labels = [], []
            for line in reader:
                feature, label = [float(i) for i in line[:-1]], int(line[-1])
                # yield feature, label
                features.append(feature)
                labels.append(label)
            return features, labels

    ds_train = tf.data.Dataset.from_tensor_slices(read_csv(r"../data/iris/iris_training.csv"))
    ds_train = ds_train.shuffle(1000).batch(16).repeat(5)
    features, labels = ds_train.make_one_shot_iterator().get_next()

    net = tf.layers.dense(features, units=10, activation=tf.nn.relu)

I'm no expert on tf.data, but according to the from_generator documentation, you can specify the shapes with the output_shapes parameter. The generator is never iterated over until a session is run, so during Graph construction time, TensorFlow has no idea what the shapes will be unless output_shapes is specified.
In your from_tensor_slices example, the shape of the input to from_tensor_slices, so TensorFlow is able to infer the shapes of the output shapes.

Thank you, I am not aware the para output_shapes. If so, I think it is better to remove the default value of this para. Actually, it took me a lot of time to find the reason, especially it work well when using tf.placeholder in my examples.


linter: Do not lint scripts parsed with tail (like tools/setup/dev-motd).

This throws an error in the console when running tools/lint as:
$ tools/lint
Error: Unknown shebang in file "tools/setup/dev-motd":
#!/usr/bin/tail -n+2

mypy      | Error: Unknown shebang in file "tools/setup/dev-motd":
mypy      | #!/usr/bin/tail -n+2
mypy      |

I’m confused. Is this failure nondeterministic? b594708915ad4c2a1975860b9dc665eb98aa7d9a passed CI, and 08f2ef089333dbd488112113a8eecec91d657ce6 only failed it on trusty.

I went on autopilot while writing the description here and forgot that this
"error" is just logged to the console and doesn't affect the return status.
I edited the description to remove that mention of it correcting a failing
build, though the latest master at that time had failed.
On Sun, May 26, 2019, 2:23 AM Anders Kaseorg<EMAIL_ADDRESS>wrote:

I’m confused. Is this failure nondeterministic? b594708
https://github.com/zulip/zulip/commit/b594708915ad4c2a1975860b9dc665eb98aa7d9a
passed CI, and 08f2ef0
https://github.com/zulip/zulip/commit/08f2ef089333dbd488112113a8eecec91d657ce6
only failed it on trusty.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/zulip/zulip/pull/12408?email_source=notifications&email_token=AB5JHVTNFUBPFZ52ZEAH7SDPXGRLXA5CNFSM4HPUW3I2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWHY7VA#issuecomment-495947732,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB5JHVS2M7LVFZ6N6Q3X3G3PXGRLXANCNFSM4HPUW3IQ
.

Ah, that makes sense.

Merged, thanks @aero31aero!


Cancel operations

commit e792f687654d564358c891c19c22b39c3eb58854
Author: Thomas Orozco<EMAIL_ADDRESS>Date:   Thu Dec 1 13:53:30 2016 +0100
Indicate operation ID when operation failed

There's a lot more detailed output coming out of Sweetness that explains
what to do, it's probably not needed for the aptible CLI to layer on its
own error message.

commit a3262998980b83d8d69cf93b45241ef74f69de53
Author: Thomas Orozco<EMAIL_ADDRESS>Date:   Thu Dec 1 13:53:22 2016 +0100
Add aptible operation:cancel OPERATION_ID

This adds the ability to cancel an operation (more specifically: to
request cancellation - only operations that support it will care for
that flag).

Ideally, I'd like this to eventually attach to the operation so you can
see it complete (and to have a matching `aptible operation:attach`
command), but this wouldn't be terribly useful right now since our
operation logging mechanism does not support having multiple clients
connected at the same time (this is perhaps something we'd fix
indirectly by sending operation logs to a Log Drain, where we have all
the infrastructure in place to deal with this already in tail drains).

cc @fancyremarker @blakepettersson

(whoops, forgot to add tests here 😄 , doing so now)

Current coverage is 88.89% (diff: 58.33%)

Merging #205 into master will decrease coverage by 0.53%

@@             master       #205   diff @@
==========================================
  Files            38         39     +1
  Lines          1202       1225    +23
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1075       1089    +14
- Misses          127        136     +9
  Partials          0          0

Powered by Codecov. Last update c69f2c9...e792f68

all done

Looks great! Thank you.


bump gateway timeout to 24 hours to match infra

What type of PR is this?

🔋 Optimization

Description
This gives our compute node 24 hours to finish a job instead of one hour. This matches the infra default timeout and allows for fire and forgetting of larger calculations.
Steps to Test
CI should be good enough to make sure it doesn't break anything. Then I can kick off up a bunch of long jobs in prod to edge test.
Relevant GIF

LGTM


Swift 4.0 update?

Hi,
Will there be a Swift 4 update for this pod?

@nzayatz14 have Pull request for this.
https://github.com/Yalantis/Segmentio/pull/54

Yes, It's already implemented and there will be updated version in several days


fix the grap x values

The graphs are repeating the same timestamp for a single minute
the image below shows 17:05 multiple times

see #46


Memory issue with Parallel Async tasks

Hi @takuya-takeuchi : I'm trying to utilize multi treads and parallel processing however it tends to consume almost all memory without freeing it. Your feedback would be greatly appreciated.
barebones.zip
public async void Button_Click_3(object sender, RoutedEventArgs e)
{
        List<string> lstPhotos = new List<string>();
        List<Task> encodeTasks = new List<Task>();
        string tempModelDirectory = "models";
        FaceRecognitionGlobal = FaceRecognition.Create(tempModelDirectory);

        for(int x=0;x<80;x++)
        {
            lstPhotos.Add(@"models\obama-480p.jpg");

        }

        foreach(string strImg in lstPhotos)
        {
              encodeTasks.Add(Task.Run(()=>TestImageASYNC(strImg)));

        }

    }//ends Button_Click_3

    public async Task TestImageASYNC(string imageToCheck)
    {

        //Tried using FaceRecognitionGlobal but I get "Access violation writing location"
        using (var FaceRecognitionFun = FaceRecognition.Create("models"))//Using this method consumes too much memory and eventually crashes
        {
            using (var unknownImage = FaceRecognition.LoadImageFile(imageToCheck))
            {
                var faceLocations = FaceRecognitionFun.FaceLocations(unknownImage, 0, Model.Cnn).ToArray();
                foreach (var faceLocation in faceLocations)
                {
                    Debug.WriteLine($"{imageToCheck},{faceLocation.Top},{faceLocation.Right},{faceLocation.Bottom},{faceLocation.Left}");

                }
            }

            // FaceRecognitionFun.Dispose();//This didn't help free up memory
        }

    }//ends TestImageASYNC

@atabora1
One point.
You should not create FaceRecognition object everytime.
This object is very heavy due to loading model files. And you should create it for each parallel thread.
However, I don't suppose working on multi thread.
Original python code looks like working fine but that is multi-process rather than multi thread.
It could help you.
https://github.com/takuya-takeuchi/FaceRecognitionDotNet/issues/24

Thanks @takuya-takeuchi, much appreciated


Generic model export pipeline + torch to ONNX and ONNX to deepsparse refactor

Summary
This PR reworks all of the onnx optimization passes contained in src/sparseml/pytorch/sparsification/quantization/quantize_qat_export.py.
The new classes/functions are introduced:

Transforms: class BaseTransform and class OnnxTransform(BaseTransform), which represent abstract transforms that act on onnx graphs
Exporters: class BaseExporter(BaseTransform), class TorchToONNX(BaseExporter), and class ONNXToDeepsparse(BaseExporter), which represent transforms that can also save things to disk

In addition, every transform from quantize_qat_export (and other files), has been added as a OnnxTransform. This includes:

The actual transform implementation
docstring of exactly what the transform does
a unit test for the transform

The get_structural_matches utility function was also added to aid in the implementation of these transforms. Notably it allows callees to match sub-graphs in any onnx graph, which makes implementation details more clear/explicit.
Test Plan
As mentioned in the summary, every transform has a unit test associated with it. Additionally, regression tests were added against the old module exporter logic for the following models:

Resnet50
Yolov5
Bert for QA
So both the graph structure and numerical output match exactly for both the new exporters and the old exporters.

@corey-nm great job! I like the PR message.
Can we override the tests and land it now?

I think @bfineran would have to force merge it


RabbitMq Connection Reset Issue

Unhandled exception. RabbitMQ.Client.Exceptions.AlreadyClosedException: Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Library, code=541, text='Unexpected Exception', classId=0, methodId=0, cause=System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException (104): Connection reset by peer
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(NetworkBinaryReader reader)
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoop()
at RabbitMQ.Client.Impl.SessionBase.Transmit(Command cmd)
at Microsoft.Azure.WebJobs.Extensions.RabbitMQ.RabbitMQListener.<>c__DisplayClass20_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.Tasks.Task.<>c.b__139_1(Object state)
at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

We have been seeing this error too under quite specific conditions, and in case it is useful I thought that I would add my findings to this ticket.
We are running a Python (currently Python 3.7) function app host (v3.17.0) with RabbitMQ extension trigger (v<IP_ADDRESS>) running in an app service image (based on 3.0-python3.7-appservice) in Kubernetes.
In our case, what appeared to be happening was that our image didn't contain the pre-compiled python scripts (i.e. the __pycache__ directories) for our functions, so these were generated on first function execution. This would trigger a restart of the JobHost by the FileMonitoringService, which would stop the RabbitMQListener while the function was still executing. So when the function finished, the RabbitMQ connection had been disconnected, which triggered the error in the ticket.
It doesn't appear to be possible to disable the file-watching behaviour of the function app host, so we think that we have resolved this by pre-compiling our python functions so that we don't trigger the JobHost restart. We'll be monitoring it to see if we still see this error.
IMO, it is unclear where the fault is here - should the function app host be restarting running functions? Or can the RabbitMQ extension somehow handle this?
There isn't much detail in the original ticket here, but just wanted to add this in case it is relevant.


Would be nice to have the ability to retain zoom with one finger as Instagram does.

🚀 Feature Proposal
When the user stops zoom with 2 fingers, currently the image comes back to its original size. Would be good to have an option that enables retain zoom size with one finger.

Amazing will there be a vuejs version?


v7.2.0: feat: add support for the projext comment on targets files

What does this PR do?
Besides updating all dependencies to their latest version (*), this PR adds a feature that allows you to write a small comment block on your target entry file with settings for projext to read... instead of writing a configuration file for a few single settings.

*: esdoc-node has a bug on their latest version, so the version here is pinned to the previous one.

Imagina you want a browser target but your target entry file doesn't use a framework nor any known browser scripts that projext may detect (document., window., $, etc.), instead of creating a configuration file to just add that type: 'browser', you can add a block like this:
/**
 * @projext
 * type: browser
 */

And projext will know that your target type is browser.
Now, this is a small feature for this kind of cases and only works for a few settings, I don't intend it to make it support all the targets settings as it doesn't make sense to mix your application with the tooling configuration.
How should it be tested manually?
Try the case explained above, and of course...
yarn test
# or
npm test

Pull Request Test Coverage Report for Build 339

25 of 25 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 336:
0.0%

Covered Lines:
2241

Relevant Lines:
2241

💛  - Coveralls


fix https://github.com/tlaverdure/laravel-echo-server/issues/85

Can you please fix the conflicts?

I fix the conflicts

Thanks!


[Garry's Mod Server] Bug GDB 'Segmentation Fault'

Hello, I have acquired a ubuntu 12.04 server, 16GB ram
and I installed a gmod server, and I decide to put the map 'rp_rockford_v1b'
I start with the command ./scrds_run
and the crash and puts me this error Server:
Engine hunk overflow!
Engine hunk overflow!
Engine hunk overflow!
Segmentation fault (core dumped)
warning: Can't read pathname for load map: Input/output error.
debug.cmds:5: Error in sourced command file:
No function contains program counter for selected frame.
email debug.log to<EMAIL_ADDRESS>Sat May  9 13:20:27 EDT 2015: Server restart in 10 seconds
Sat May  9 13:20:30 EDT 2015: Server Quit
Can you help me?

If it's an option update to 14.04, much more stable and I've never had problems with srcds on it.

Yes im afraid something funky is happening with gmod server and your distro and it may not be easy to fix. I recommend Googling around to see if there is anything on this error. Failing that a newer distro like 14.04 LTS is a good idea.

I should actually point out that Ubuntu 15.04 was released, better to use that over 14.04.

14.04 is long term support. Which I normally a better idea for servers

I've never generally had a problem with the latest version of Ubuntu when compared against a LTS version, I guess I've just never ran across a problem where LTS would have been better to have.


multiprocessing error

miniforge3/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

simple code:
from g2pw import G2PWConverter

conv = G2PWConverter(style='pinyin', enable_non_tradional_chinese=True)
conv('然而，他红了20年以后，他竟退出了大家的视线。')

try to use python3.7 ?


Handling XML payload

Hi and thanks for a very nice tool!
I have been trying to set up the webhook to accept an XML payload. So far I am failing quite miserably. I have tried to just pass on the entire payload both as an environment variable or a file but it is failing and just producing a 'null' content.
If I do a POST without a content type header set, then I do seem to get the xml payload output as an escaped JSON string.... not really what I am hoping for.
Is there a reason why really only JSON seems to be supported?
Thanks,
Daniel

Hey Daniel, thanks for writing up. The reason why only JSON (and formdata) seems to be supported is that I never needed to ingest such payload.
Adding XML right now would require a bit of thought how to fit it in properly. (i.e. using xpath for referencing values, etc...)
Would adding support to webhook to pass the raw request payload without attempting to parse it help you in any way? But then again, what benefits would webhook provide to you in that case :-(

Dear Adnan,
Thank you for your reply. Unfortunately we have already moved towards
another solution, but yes being able to pass the raw request paylout would
have been sufficient as far as I remember!
Best wishes,
Daniel
On Thu, Apr 12, 2018 at 11:41 AM, Adnan Hajdarević<EMAIL_ADDRESS>
wrote:

Hey Daniel, thanks for writing up. The reason why only JSON (and formdata)
seems to be supported is that I never needed to ingest such payload.
Adding XML right now would require a bit of thought how to fit it in
properly. (i.e. using xpath for referencing values, etc...)
Would adding support to webhook to pass the raw request payload without
attempting to parse it help you in any way? But then again, what benefits
would webhook provide to you in that case :-(
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/adnanh/webhook/issues/238#issuecomment-380742425, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AcgmezXjBeDgnzzGaOsebHbeS3YIxqihks5tnyFSgaJpZM4SYuzh
.

I would be interested in XML support but in mean time I can transform XML to JSON as needed.

Hello Adnan Hajdarević , Do you have timeline when webhook will pass the raw request payload???
Thanks
Osmar

My team would sincerely appreciate the ability to capture a raw payload without attempting to parse it.

I am working on a feature which would possibly allow you to do this. It involves writing a custom script which would accept raw payload, headers etc, and then output a json object which you would be able to access and pass down. You can try it out by compiling webhook from the feature/context-provider-command branch. There's no documentation at the moment, as I intend doing a RFC before pushing it out, but it works by specifying a "context-provider-command" field in the hook description. The script will be passed an JSON string with an object containing various raw values. The output of the command has to be valid JSON string which will be parsed back by the webhook, and you can access those fields by specifying source as "context" instead of the usual "payload", "query", etc...


DOC: 9789 Added missing letter, fixed link and Examples formatting.

Partial fix for https://github.com/pydata/pandas/issues/9789.
How do you feel about direct URLs in the doctrings?
I saw them in a few other docstrings. Pandas maintainers, any preferences?
Still need to fix the Examples at the bottom of the page.

lgtm. is there a standard to putting links in doc-strings so that they render when viewed in html (and text otherwise)? @jorisvandenbossche

@jorisvandenbossche @jreback
I found some examples in thepandas API docs where links are rendered with nicer markup and the format seems to be Link text <URL>_. This also follows the Sphinx documentation style guide for links and referencesso unless you have any objections, I'll change the markup in my pull to reflect the Sphinx recommendations.

Hope the squashing worked.

About the url. In principle this can even be made shorter by using the internal links. Here, this would be something like:
:ref:`here <timeseries>`

The disadvantage of this is that when looking at the docstring in text format (in completion box eg) and not online, you don't see a url you can copy paste.
So I think in this case your approach is a good compromise: use the full url (for when looking in text format), but with a shorter version in the html docs.

thanks!


dirPagination: on-page-change called 6 times each time I change page (version 0.9.4)

I don't think it used to happen, but now I see that every time I change page the on-page-change event is triggered anything from 2 to 6 times resulting in slow list updates as each time it is called I refresh the page from the server (I am using server paging)
When debugging the code
  scope.$watch(function() {
                if (paginationService.isRegistered(paginationId)) {
                    return paginationService.getCurrentPage(paginationId);
                }
            }, function(currentPage, previousPage) {
                if (currentPage != previousPage) {
                    goToPage(currentPage);
                }
            });

triggers the goToPage function multiple times when I change from one page to another
all the variables paginationId, currentPage and previousPage as as expected, it just calls the goToPage multiple times

Hi,
Please could you fork and edit this plunker demo and try to reproduce your issue? Paste the link to your updated demo here so I can take a look.
Note that the latest version is 0.11.0 - although I'm not sure if that should make a difference.

I have recreated on plunker... here is the link
[http://plnkr.co/edit/wMSxBaJgmQ94QoOEySwz]
I know what the issue is now.... but I dont know how to fix it
I like to have 2 pagination controls, one above and one below the list, which is fine as I use the pagination-id to keep everything in sync. Because each pagination control has an on-page-change attribute, you will see that each time I change the page the event is fired twice which in my real life case, causes to hits to my API (I am using server paging in my real life case). If I were to remove the on-page-change attribute from one of the pagination controls then it all works correctly. However I prefer not to have to do that because in my real life case I use different pagination templates for different screen sizes and my code is more like this plunk (albeit with the pagination controls wrapped up in my own directive)
[http://plnkr.co/edit/wbNOx6OLo79GPgU3CUpf?p=info]
Now we get at least 4 hits to the API each time I change page...see the console log
I hope you understand my issue and can point me in the right direction
Many thanks in advance

Hi,
Thanks for the clear demos. I have a few suggestions:

If you were to use ng-if on your controls directives, the extra callbacks would not fire. In your case this might not be desirable since you are reacting to screen size, and tracking that in JS may be too much of a burden.
You can safely remove all but one of the on-page-change attributes and, assuming the one with the attribute set is in your view, it will fire when any of the linked controls are clicked.
The best way to go might be to get rid of that callback altogether and instead just use a $scope.$watch on the value of currentPage. This means that it does not matter how you change the page - it will only invoke your page change function once. I modified your last demo to show this: http://plnkr.co/edit/lM8UKlPwSWoiFB4C364U?p=preview

I think option 3 is the way ahead... many thanks I can make that work :)

Alternatively, I might check that the page has changed by storing the last page requested in a variable and block calls that are not genuinely for a new page

+1

@hplloyd Yes, that's a good suggestion. I'm going to leave this issue open with an "enhancement" label to remind myself to consider implementing when I make the next update to the lib (no idea when that will be at present).

No worries, my fix is working fine for me until you implement :)

On 19 May 2016, at 5:48 PM, Michael Bromley<EMAIL_ADDRESS>wrote:
@hplloyd https://github.com/hplloyd Yes, that's a good suggestion. I'm going to leave this issue open with an "enhancement" label to remind myself to consider implementing when I make the next update to the lib (no idea when that will be at present).
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub https://github.com/michaelbromley/angularUtils/issues/324#issuecomment-220251897

Any news regarding this issue? Or is it still the easiest way to solve this issue the way @hplloyd solved it?

@GregaVrbancic For now I have made not incorporated a fix into the code. Just go with one of the work-arounds mentioned above. To be honest, this module is not something I require for my work projects, and is therefore not high on my list of priorities for spending time on.


add link verification steps

Cleaning up all the bad (error 404) links in .NET repo
Enable link validation as part of PR and aggregate pipeline

Aggregate Reports Check - https://dev.azure.com/azure-sdk/internal/_build/results?buildId=482219&view=results

/check-enforcer override


Dissmiss a review on a pull request and input the reason but the reason missed on the conversation.

May be related to https://github.com/go-gitea/gitea/pull/24526, I'm not sure.

At least the rendering has not changed, so it might be a problem when the dismissal request is sent.
https://github.com/go-gitea/gitea/blame/main/templates/repo/issue/view_content/comments.tmpl#L76

The input form is pretty much pure copy-paste, so I think it's unlikely that that PR regressed it and this may be a older bug.

It seems the bug exists from the beginning of the "dismiss" feature.
-> Make "dismiss" content shown correctly #25461


Support for Rendering in MJX for Simulated Camera Reinforcement Learning

import jax
import time
import mujoco
import functools
from brax import envs
from mujoco import mjx
from etils import epath
from jax import numpy as jp
from brax.io import mjcf, model
from brax.envs.base import PipelineEnv, State
from brax.training.agents.ppo import train as ppo

class Humanoid(PipelineEnv):

    def __init__(self, **kwargs):
        mj_model = mujoco.MjModel.from_xml_path((epath.Path(epath.resource_path("mujoco")) / ("mjx/test_data/humanoid") / "humanoid.xml").as_posix())
        mj_model.opt.solver = mujoco.mjtSolver.mjSOL_CG
        mj_model.opt.iterations = 6
        mj_model.opt.ls_iterations = 6

        sys = mjcf.load_model(mj_model)

        physics_steps_per_control_step = 5
        kwargs["n_frames"] = kwargs.get("n_frames", physics_steps_per_control_step)
        kwargs["backend"] = "mjx"

        super().__init__(sys, **kwargs)

    def reset(self, rng: jp.ndarray) -> State:
        """Resets the environment to an initial state."""
        rng, rng1, rng2 = jax.random.split(rng, 3)

        low, hi = -1e-2, 1e-2 # reset_noise_scale
        qpos = self.sys.qpos0 + jax.random.uniform(rng1, (self.sys.nq,), minval=low, maxval=hi)
        qvel = jax.random.uniform(rng2, (self.sys.nv,), minval=low, maxval=hi)

        data = self.pipeline_init(qpos, qvel)

        obs = self._get_obs(data, jp.zeros(self.sys.nu), None)
        reward, done, zero = jp.zeros(3)
        metrics = {
            "forward_reward": zero, "reward_linvel": zero, "reward_quadctrl": zero, "reward_alive": zero,
            "x_position": zero, "y_position": zero,"distance_from_origin": zero,"x_velocity": zero,"y_velocity": zero
        }
        return State(data, obs, reward, done, metrics)

    def step(self, state: State, action: jp.ndarray) -> State:
        """Runs one timestep of the environment's dynamics."""
        data0 = state.pipeline_state
        data = self.pipeline_step(data0, action)

        com_before = data0.subtree_com[1] # center of mass
        com_after = data.subtree_com[1]
        velocity = (com_after - com_before) / self.dt
        forward_reward = 1.25 * velocity[0]

        min_z, max_z = (1.0, 2.0) # healthy_z_range
        is_healthy = jp.where(data.q[2] < min_z, 0.0, 1.0)
        is_healthy = jp.where(data.q[2] > max_z, 0.0, is_healthy)
        healthy_reward = 5.0
        ctrl_cost = (0.1 * jp.sum(jp.square(action)))

        state.metrics.update(
            forward_reward=forward_reward, reward_linvel=forward_reward, reward_quadctrl=-ctrl_cost,
            reward_alive=healthy_reward, x_position=com_after[0], y_position=com_after[1],
            distance_from_origin=jp.linalg.norm(com_after), x_velocity=velocity[0], y_velocity=velocity[1]
        )
        return state.replace(
            pipeline_state=data, obs=self._get_obs(data, action, state),
            reward=(forward_reward + healthy_reward - ctrl_cost), done=(1.0 - is_healthy)
        )

    def _get_obs(self, data: mjx.Data, action: jp.ndarray, state) -> jp.ndarray:
        """Observes humanoid body position, velocities, and angles."""
        if state is None:
          pixels = jax.numpy.zeros((240,320)).ravel()
        else:
          pixels = jax.numpy.array(self.render(state.pipeline_state, camera="egocentric")).ravel()

        return jp.concatenate([data.qpos[2:], data.qvel, data.cinert[1:].ravel(), data.cvel[1:].ravel(), data.qfrc_actuator, pixels])

train_fn = functools.partial(
    ppo.train, num_timesteps=3_000_000, num_evals=5, reward_scaling=0.1, episode_length=1000, normalize_observations=True,
    action_repeat=1, unroll_length=10, num_minibatches=16, num_updates_per_batch=8, discounting=0.97, learning_rate=3e-4, entropy_cost=1e-3,
    num_envs=1024, batch_size=512, seed=0
)

start_time = time.time()

print(f"Registering environment... ({time.time() - start_time:.2f}s)")
envs.register_environment("humanoid_mjx", Humanoid)

print(f"Starting training... ({time.time() - start_time:.2f}s)")
make_inference_fn, params, _ = train_fn(environment=envs.get_environment("humanoid_mjx"))

print(f"Training completed. Saving parameters... ({time.time() - start_time:.2f}s)")
model.save_params("/home/markusheimerl/mjx_brax_policy_v2", params)

print(f"Parameters saved. ({time.time() - start_time:.2f}s)")

results in:
(venv_mjx) markusheimerl@tower:~/sim/hum$ python brax_humanoid.py
Registering environment... (0.00s)
Starting training... (0.00s)
Traceback (most recent call last):
  File "/home/markusheimerl/sim/hum/brax_humanoid.py", line 95, in <module>
    make_inference_fn, params, _ = train_fn(environment=envs.get_environment("humanoid_mjx"))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/training/agents/ppo/train.py", line 405, in train
    metrics = evaluator.run_evaluation(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/training/acting.py", line 125, in run_evaluation
    eval_state = self._generate_eval_unroll(policy_params, unroll_key)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/training/acting.py", line 107, in generate_eval_unroll
    return generate_unroll(
           ^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/training/acting.py", line 75, in generate_unroll
    (final_state, _), data = jax.lax.scan(
                             ^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/training/acting.py", line 71, in f
    nstate, transition = actor_step(
                         ^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/training/acting.py", line 43, in actor_step
    nstate = env.step(env_state, actions)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/wrappers/training.py", line 176, in step
    nstate = self.env.step(state, action)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/wrappers/training.py", line 122, in step
    state = self.env.step(state, action)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/wrappers/training.py", line 71, in step
    return jax.vmap(self.env.step)(state, action)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/wrappers/training.py", line 93, in step
    state, rewards = jax.lax.scan(f, state, (), self.action_repeat)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/wrappers/training.py", line 90, in f
    nstate = self.env.step(state, action)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/sim/hum/brax_humanoid.py", line 69, in step
    pipeline_state=data, obs=self._get_obs(data, action, state),
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/sim/hum/brax_humanoid.py", line 78, in _get_obs
    pixels = jax.numpy.array(self.render(state.pipeline_state, camera="egocentric")).ravel()
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/base.py", line 159, in render
    return image.render_array(self.sys, trajectory, height, width, camera)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/io/image.py", line 48, in render_array
    return get_image(trajectory)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/io/image.py", line 40, in get_image
    d.qpos, d.qvel = state.q, state.qd
    ^^^^^^
jax.errors.TracerArrayConversionError: The numpy.ndarray conversion method __array__() was called on traced array with shape float32[28].
The error occurred while tracing the function f at /home/markusheimerl/venv_mjx/lib/python3.12/site-packages/brax/envs/wrappers/training.py:89 for scan. This concrete value was not available in Python because it depends on the value of the argument state.pipeline_state.q.
See https://jax.readthedocs.io/en/latest/errors.html#jax.errors.TracerArrayConversionError
--------------------
For simplicity, JAX has removed its internal frames from the traceback of the following exception. Set JAX_TRACEBACK_FILTERING=off to include these.

Description:
I'm encountering an issue while using MuJoCo with JAX (mjx) for training a humanoid model in the Brax environment. The problem arises when attempting to render the environment state and retrieve camera images during training. The MuJoCo renderer does not seem to work properly when using mjx.
Problem Details:
When calling the render method within the custom Humanoid class, an error is thrown:
jax.errors.TracerArrayConversionError: The numpy.ndarray conversion method __array__() was called on traced array with shape float32[28].
This error indicates that the conversion method is called on a traced array, which depends on the value of the argument state.pipeline_state.q. The current implementation of the MuJoCo renderer in mjx does not handle the conversion of traced arrays properly in this context.
Importance of Camera Input in Reinforcement Learning:
Using camera input is crucial when training robots with reinforcement learning. In real-world scenarios, robots rely on visual information captured by their cameras to perceive and interact with the environment. By incorporating camera pixels as part of the observation space during training, the learned policies can be more robust and adaptable to real-world conditions.
Proposed Solution:
To enable effective training with camera input using mjx, it is essential to address the compatibility issue between the MuJoCo renderer and JAX traced arrays. Possible solutions include:

Enhancing the mjx renderer to handle JAX traced arrays directly, ensuring seamless integration with the Brax environment.
Providing alternative rendering methods or workarounds specifically designed for mjx, allowing the retrieval of camera images during training.
Improving documentation and examples that demonstrate the proper usage of camera input with mjx and provide guidance on handling traced arrays in this context.

Alternatives Considered:

Preprocessing the camera images outside of the JAX computation graph, but this approach may introduce additional overhead and complexity to the training pipeline.
Using alternative rendering libraries or techniques, but this may limit the flexibility and compatibility with the existing MuJoCo and Brax ecosystem.

Additional Context:
Integrating camera input into reinforcement learning algorithms is crucial for developing intelligent and adaptable robots. By leveraging the power of JAX and mjx, researchers and developers can accelerate the training process and build more sophisticated models. However, the current compatibility issue between the MuJoCo renderer and JAX traced arrays hinders the effective utilization of camera input in this setup.
Addressing this issue and providing a seamless integration between the MuJoCo renderer and mjx will greatly benefit the robotics and reinforcement learning community. It will enable researchers to train models that can effectively process visual information, leading to more advanced and capable robots.
Thank you for considering this issue. Your support in resolving the compatibility problem and enhancing the usability of camera input with mjx will contribute to the advancement of robotics research and real-world applications.

You are 100% right.
We are working on option 2.

@yotam you have an estimate of when this might be done?

@yuvaltassa I think what I wanted to ask: Would you mind making the corresponding feature branch public in this repo? Id love to contribute.

The moment we have something that works it will be OSS and we would love you to contribute!
@erikfrey is leading this effort, perhaps there is something he'd like to add.

Hello!  Please see #1604 and #1485 for related discussions.  You can do visual observations today using mjx.ray, although this only works for toy environments.
We are working on integrating Madrona as a means for high throughput tiled rendering on GPU, but this is still very much a work in progress.  We'll share more once we have a good proof of concept - no ETA but this is actively under development.

@yuvaltassa @erikfrey thanks for your input! Have you considered https://github.com/JoeyTeng/jaxrenderer ?
import re
import jax
import numpy as onp
from PIL import Image
from jax import numpy as jp
from renderer import Model
from renderer import ModelObject
from renderer import LightParameters
from renderer.geometry import rotation_matrix
from renderer import CameraParameters
from renderer import ShadowParameters
from renderer import Renderer, transpose_for_display
from numpngw import write_apng

# Load model and textures
obj_path, texture_path, spec_path = "african_head.obj", "african_head_diffuse.tga", "african_head_spec.tga"
image = Image.open(texture_path)
width, height = image.size
texture = onp.zeros((width, height, 3))
for y in range(height):
    for x in range(width):
        texture[y, x] = onp.array(image.getpixel((x, y)))
texture = jp.array(texture, dtype=jp.single) / 255

image = Image.open(spec_path)
specular_map = onp.zeros((width, height, 3))
for y in range(height):
    for x in range(width):
        specular_map[y, x] = onp.array(image.getpixel((x, y)))
specular_map = jp.array(specular_map, dtype=jp.single)[..., 0]

verts, norms, uv, faces, faces_norm, faces_uv = [], [], [], [], [], []
_float, _integer, _one_vertex = re.compile(r"(-?\d+\.?\d*(?:e[+-]\d+)?)"), re.compile(r"\d+"), re.compile(r"\d+/\d*/\d*")

with open(obj_path, 'r') as file:
    for line in file:
        if line.startswith("v "):
            verts.append(tuple(map(float, _float.findall(line, 2)[:3])))
        elif line.startswith("vn "):
            norms.append(tuple(map(float, _float.findall(line, 2)[:3])))
        elif line.startswith("vt "):
            uv.append(tuple(map(float, _float.findall(line, 2)[:2])))
        elif line.startswith("f "):
            face, face_norm, face_uv = [], [], []
            vertices = _one_vertex.findall(line)
            assert len(vertices) == 3, f"Expected 3 vertices, got {len(vertices)}"
            for vertex in vertices:
                v, vt, vn = list(map(int, _integer.findall(vertex)))
                face.append(v - 1)
                face_norm.append(vn - 1)
                face_uv.append(vt - 1)
            faces.append(face)
            faces_norm.append(face_norm)
            faces_uv.append(face_uv)

model = Model(
    verts=jp.array(verts),
    norms=jp.array(norms),
    uvs=jp.array(uv),
    faces=jp.array(faces),
    faces_norm=jp.array(faces_norm),
    faces_uv=jp.array(faces_uv),
    diffuse_map=jax.numpy.swapaxes(texture, 0, 1)[:, ::-1, :],
    specular_map=jax.numpy.swapaxes(specular_map, 0, 1)[:, ::-1],
)

canvas_width, canvas_height, frames, rotation_axis = 1920, 1080, 30, "Y"
rotation_axis = dict(X=(1., 0., 0.), Y=(0., 1., 0.), Z=(0., 0., 1.))[rotation_axis]
degrees = jax.lax.iota(float, frames) * 360. / frames

eye, center, up = jp.array((0, 0, 3.)), jp.array((0, 0, 0)), jp.array((0, 1, 0))
camera = CameraParameters(viewWidth=canvas_width, viewHeight=canvas_height, position=eye, target=center, up=up)
light = LightParameters(direction=jp.array([0.57735, -0.57735, 0.57735]), ambient=0.1, diffuse=0.85, specular=0.05)
shadow = ShadowParameters(centre=center)

@jax.default_matmul_precision("float32")
def render_instances(instances, width, height, camera, light, shadow):
    img = Renderer.get_camera_image(objects=instances, light=light, camera=camera, width=width, height=height, shadow_param=shadow, colour_default=jp.zeros(3, dtype=jp.single))
    return jax.lax.clamp(0., img, 1.)

def rotate(model, rotation_axis, degree):
    instance = ModelObject(model=model)
    return instance.replace_with_orientation(rotation_matrix=rotation_matrix(rotation_axis, degree))

batch_rotation = jax.jit(jax.vmap(lambda degree: rotate(model, rotation_axis, degree))).lower(degrees).compile()
instances = [batch_rotation(degrees)]

@jax.jit
def render(batched_instances):
    def _render(instances):
        _render = jax.jit(render_instances, static_argnames=("width", "height"), inline=True)
        img = _render(instances=instances, width=canvas_width, height=canvas_height, camera=camera, light=light, shadow=shadow)
        return transpose_for_display((img * 255).astype(jp.uint8))

    return jax.jit(jax.vmap(_render))(batched_instances)

render_compiled = jax.jit(render).lower(instances).compile()
images = list(map(onp.asarray, jax.device_get(render_compiled(instances))))

write_apng('animation.png', images, delay=1/30.)
# ffmpeg -i animation.png intermediate.gif
# gifsicle --optimize=3 --delay=5 intermediate.gif > output.gif

All these views were rendered in parallel using jax as the only dependency:

Thank you for your input. Godspeed on integrating Madrona.

@markusheimerl in case you didn't see it already, here seems to be a working approach of using madrona for rendering in mjx:
https://github.com/shacklettbp/madrona_mjx


Some sources fail to install

See, for example: https://travis-ci.org/travis-ci/apt-whitelist-checker/builds/72284682#L401
W: Failed to fetch http://downloads-distro.mongodb.org/repo/ubuntu-upstart/dists/dist/Release  Unable to find expected entry '10gen/source/Sources' in Release file (Wrong sources.list entry or malformed file)

Hmm. I added the tests to add all keys, and they're successful. https://travis-ci.org/travis-ci/apt-source-whitelist/builds/72345700

Actually, the error messages are coming from the subsequent run of sudo apt-get update.

The number of warning messages seem to vary if we rerun them, but I guess there is not much we can do to actually fix these errors.


[CLOSED] Add the Medical Safety Alert specialist document format

 Issue by issyl0
Thursday 11 February 2016 at 11:33 GMT
Originally opened as https://github.com/alphagov/specialist-publisher/pull/646

Add a schema for it, metadata form fields, a model and its
validations, and model specs.

https://trello.com/c/UFpwZWOS/537-msa

issyl0 included the following code: https://github.com/alphagov/specialist-publisher/pull/646/commits

 Comment by boffbowsh
Tuesday 16 February 2016 at 10:15 GMT

See the changes in https://github.com/alphagov/specialist-publisher/pull/655 for the style of these specs

 Comment by issyl0
Wednesday 17 February 2016 at 17:26 GMT

Thanks @tommyp, @boffbowsh. I'll get round to updating the specs tomorrow, promise. :sunny:

 Comment by issyl0
Thursday 18 February 2016 at 12:56 GMT

@tommyp @boffbowsh I've changed the style of the specs as per #656.

 Comment by Rosa-Fox
Monday 22 February 2016 at 13:35 GMT

@issyl0 Teeny tiny thing which is that ".save!" on 135 of medial_safety_alert_spec.rb should be "#save!"

 Comment by issyl0
Monday 22 February 2016 at 14:56 GMT

@Rosa-Fox Oh! Thanks. Fixed.


[UX] It's often hard to reach the footer

STR
Open e.g. https://heisse-preise.io/changes.html?f=-;-;-;-;-;-;-;-;-;-;-;-;-;-;100;0;-;2023-09-18;.;.;.;&l=-;.;store-and-name;-&c=2023-09-18;-;-;2017-01-01;-;-&d= and try to scoll down to click/open the footer.
What happens
The bad thing is you see it happening and you see the footer, but it is "so close, but out of reach", because you cannot click it.
Solution

either have no footer, but things somewhere else, like all social media sites
or pin/"stick" it to the bottom, so it is always visible (could be too big especially on mobiles e.g. though)

It's unlikely you want to access the footer while scrolling a large result lost. As such I think the current UX is fine, as the footer is easily accessible otherwise.


Fix map view on web, map displays "for development purposes only" watermarks

This issue is related to Googles changes to the billing system.
See, https://stackoverflow.com/questions/50977913/google-maps-shows-for-development-purposes-only

Google Maps is no longer free. You have to associate a credit card so that you can get billed if your site has requests that exceed the $200 credit they give you monthly for free. That is why you get the watermarked maps.

Issue not relevant to app functions but rather google account, therefore not necessary


Rename Constitutional Committee certificates

Current names for Committee certificates are:
https://github.com/input-output-hk/cardano-ledger/blob/c1603995da7c57d95371844191d235b05bb476b7/eras/conway/impl/src/Cardano/Ledger/Conway/TxCert.hs#L147-L148
These should be renamed to:
data ConwayCommitteeCert c
  = ConwayAuthCommitteeHotKey !(KeyHash 'CommitteeColdKey c) !(KeyHash 'CommitteeHotKey c)
  | ConwayResignCommitteeColdKey !(KeyHash 'CommitteeColdKey c)

This rename should also apply to pattern synonyms implemented in #3455

Closed by https://github.com/input-output-hk/cardano-ledger/pull/3462


feat: add optional created_at and initialized_at for rw relations

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
dev=> select name,initialized_at,created_at from rw_sources;
 name |        initialized_at         |          created_at
------+-------------------------------+-------------------------------
 s    | 2023-07-25 10:53:30.128+00:00 | 2023-07-25 10:53:30.130+00:00
(1 row)

dev=> select name,initialized_at,created_at from rw_sinks;
 name |        initialized_at         |          created_at
------+-------------------------------+-------------------------------
 ss   | 2023-07-25 10:53:32.176+00:00 | 2023-07-25 10:53:43.238+00:00
(1 row)

dev=> select name,initialized_at,created_at from rw_tables;
 name |        initialized_at         |          created_at
------+-------------------------------+-------------------------------
 t    | 2023-07-25 10:53:28.030+00:00 | 2023-07-25 10:53:28.052+00:00
(1 row)

dev=> select name,initialized_at,created_at from rw_indexes;
 name |        initialized_at         |          created_at
------+-------------------------------+-------------------------------
 x    | 2023-07-25 10:53:30.135+00:00 | 2023-07-25 10:53:32.171+00:00
(1 row)

dev=> select name,initialized_at,created_at from rw_materialized_views;
 name |        initialized_at         |          created_at
------+-------------------------------+-------------------------------
 m    | 2023-07-25 10:53:28.089+00:00 | 2023-07-25 10:53:30.123+00:00
(1 row)

dev=> select relationname, relationtype , initialized_at, created_at from rw_relation_info ;
 relationname |   relationtype    |        initialized_at         |          created_at
--------------+-------------------+-------------------------------+-------------------------------
 m            | MATERIALIZED VIEW | 2023-07-25 10:53:28.089+00:00 | 2023-07-25 10:53:30.123+00:00
 t            | TABLE             | 2023-07-25 10:53:28.030+00:00 | 2023-07-25 10:53:28.052+00:00
 ss           | SINK              | 2023-07-25 10:53:32.176+00:00 | 2023-07-25 10:53:43.238+00:00
 x            | INDEX             | 2023-07-25 10:53:30.135+00:00 | 2023-07-25 10:53:32.171+00:00
 s            | SOURCE            | 2023-07-25 10:53:30.128+00:00 | 2023-07-25 10:53:30.130+00:00
(5 rows)

Checklist

[x] I have written necessary rustdoc comments
[x] I have added necessary unit tests and integration tests
[x] All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

cc @neverchanje


Interpolation tests changed

Changing tests due to modifications in openvino
https://github.com/openvinotoolkit/openvino/pull/5520

@ilya-lavrenov could you please take a look?


Add secret type

This PR introduces a new type Secret which aims to
better encapsulate handling of secrets, especially
one containing YAML.

Yes, of course. It's probably not obvious why this makes sense w/o having dug too deep into the code/history of the project.
In the past we introduced several features on top of regular plain text secrets. However many of those features, like YAML, needed to be implemented spread all over the place. This made testing and modifications very hard and introduced a whole bunch of bugs over time. This PR is a first attempt to better manage these kinds of features by putting an abstraction on top of the plain text secrets. I'm not yet fully happy with the interface design, but it's an IMHO very good first step upon we can easily build further improvements.


Update presentation.html

Made changes to the presentation.html. Just a content update

Made changes to the presentation.html. Just a content update


UHF-8509 CKEditor5 table

UHF-8509

What was done

This thing was fixed

How to install

Make sure your instance is up and running on latest dev branch.

git pull origin dev
make fresh

Update the Helfi Platform config

composer require drupal/helfi_platform_config:dev-UHF-8509_ckeditor5_table

Run make drush-updb drush-cr

How to test

[ ] Check that this feature works
[ ] Check that code follows our standards

Other PRs

Link to other PR

Codecov Report

Merging #619 (073d104) into ckeditor5 (e0a7899) will not change coverage.
Report is 6 commits behind head on ckeditor5.
The diff coverage is n/a.

:exclamation: Current head 073d104 differs from pull request most recent head 23b5fb3. Consider uploading reports for the commit 23b5fb3 to get more accurate results

@@             Coverage Diff              @@
##             ckeditor5     #619   +/-   ##
============================================
  Coverage        12.74%   12.74%
  Complexity         236      236
============================================
  Files               30       30
  Lines              902      902
============================================
  Hits               115      115
  Misses             787      787

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Support 8 uvs

Describe the proposed feature and how it helps to overcome a problem or limitation
Godot supports 8 uvs. So it'll be good for godot-ufbx to support 8 uvs for consistency.
Other information about your context or usecase.
8uvs are commonly used in shaders
If known, describe how your proposal will work, with code, pseudo-code, mock-ups, or diagrams
To be determined.
Anything else
No response

Completed as of https://github.com/V-Sekai/godot-ufbx/commit/26805bcb7f9f12cb74b1b1ae2189169da829326a


setting the auto binded queue to auto-delete

Is it possible to set the feature of the auto binded queue to auto-delete?
In my setup I only have one exchange and topic based routing. Currently if their is no consumer left for a queue, the queue will not be deleted and gets full of messages no one may ever read.

@marvinrichter Depending on your needs, if you set it as an anonymous consumer (no consumer group) then the queue will be automatically deleted. This wouldn't work if you require multiple competing consumers. If you do need a consumer group then you can use the durableSubscription setting - see http://docs.spring.io/autorepo/docs/spring-cloud-stream-docs/Brooklyn.BUILD-SNAPSHOT/reference/htmlsingle/#_rabbitmq_consumer_properties

Ahh cool thanks durableSubscription was the right config.


type inference on promises leaks to unrelated promises

The following code gives an unexpected error:
'use strict' // @flow

function f(): Promise<number> {
  return Promise.resolve(1)
}

function g(): any {
  let promise = Promise.resolve(null)
  if (Math.random() > 0.5) {
    promise = f()
  }

  return promise.then(() => {
    return 1
  })
}

The error I get is:
test.js:8
  8:   let promise = Promise.resolve(null)
                     ^^^^^^^^^^^^^^^^^^^^^ call of method `resolve`
  8:   let promise = Promise.resolve(null)
                                     ^^^^ null. This type is incompatible with
  3: function f(): Promise<number> {
                           ^^^^^^ number

Bizarrely, if i inline the function f, the problem goes away.
If I remove the then at the bottom of the function, the problem also goes away.
It happens when I tried to upgrade from flow 0.24 -> 0.28. It was not reproducible in 0.24.

@nicks This is a technical limitation, due to an approximation Flow has to make for unrelated technical reasons (avoiding non-termination). I'm afraid the explanation is complicated, but thankfully the workaround is simple.
When looking at a method call like then on a variable like promise whose type could be Promise<T1>, Promise<T2>, etc., Flow assumes that T1 = T2 = ....
Here, the two different instantiations of Promise that promise can have is confusing Flow. You can avoid the confusion by annotating promise with (say) Promise<?number>.
The reason inlining works is that Promise.resolve(1) doesn't return a Promise<number>, but some Promise<X> where number is a subtype of X. In that scenario there is no conflict...X can be wide enough to contain both number and null, in particular it can be ?number. It is pinning down the instantiation via the return type of f that causes the confusion.

:+1:


Generate trx from scratch (WIP)

Here is a script to build a TRX from a collection of NPY. This script will assemble/cast the NPY into a valid TRX file.
This is still a work in progress, but I have example data here.
A reference, positions (CSV), or positions+offsets must be provided. As much metadata as desired can be added.
Due to the support in data type and for simplicity, this script does load arrays in memory.
@anibalsolon When the data is uploaded to BrainLife and the fetcher modified in the code to replace the GDrive.
Could I get a small tutorial on how to add data to BrainLife and fetch it?
I will keep working on the script and improve edge-case and docstring until there is a PR to replace the GDown fetcher by BrainLife. I have a simple test in mind to verify if this script works as intended, but I will wait for issue #25 to be solved.

Hey @frheault
BL currently provides with a CLI tool as programming interface.To use the CLI, you can follow the instructions here: https://brainlife.io/docs/cli/install/
TL'DR: install Node.js and npm i -g brainlife
A Python implementation of CLI and lib to interact with BL may arrive in the next months 🤞🏻
I've set-up in the org's secrets the BL user and password created for TRX use. So to log in in the CLI tool, you use:
bl login --username=$BL_USERNAME --password=$BL_PASSWORD

This is a small script on how to fetch data:
# Project ID
PID=6289982fd0697cf1eae8e88d

# subjects: sub-complete, sub-medium, sub-one, sub-resize, sub-small
# datatypes: neuro/track/trx, neuro/track/tck, neuro/track/trk
ID=$(
    bl data query \
        -p $PID \
        -d neuro/track/trx \
        --subject sub-complete \
        --json | jq -r '.[] | ._id'
)

# Download data
bl data download -i $ID -d data

# Move the file to a more meaningful name (and to avoid overwrite)
mv data/track.trx data/complete.trx

and now to upload, pretty easy:
# Project ID
PID=6289982fd0697cf1eae8e88d

bl data upload --datatype neuro/track/tck --project $PID --subject sub-0001 --track track.tck
bl data upload --datatype neuro/track/trk --project $PID --subject sub-0001 --track track.trk
bl data upload --datatype neuro/track/trx --project $PID --subject sub-0001 --track track.trx

Can I suggest we share TRX data sets using public repositories that do not require a password? Some datasets could be used for regression testing. For example, every commit to dcm2niix uses AppVeyor and Travis to explicitly check conversion of the data in the dcm_qa, dcm_qa_nih, dcm_qa_uih submodules. Each dcm_qa tests different features (e.g. manufacturer). One limitation of Github repositories are file size limits. For larger datasets, I use OSF which allows you to include huge datasets and a wiki to describe the rationale.
If there is a concern about sharing raw DTI data due to HIPAA, consider using one of the public DTI datasets. The dcm2niix wiki provides examples from Siemens, Philips and GE.

+1 for @neurolabusc suggestions. I agree that we should avoid using password.
He might also be over complicated if an user want to test it locally with the provided dataset

Yeah. Fair enough.
@francopestilli : is there a way to create a dataset that don't require authentication on brainlife? Otherwise, I think that we can also use zenodo or figshare for this data.

Maybe let's go ahead and upload benchmark data to figshare for now? We can always shift to brainlife if it becomes possible to get the data without authentication.

hi @arokem see ehre: https://brainlife.io/docs/cli/download/
We require login. This is because the data, well contains brains! :-)
what is the issue with login?
We can create a project, add our users and do one fo the following
See examples:
$ bl data download -i 5a050ad5d76a2a002737e572
Download dataset to 5a050ad5d76a2a002737e572
100% done

$ ls 5a050ad5d76a2a002737e572
t1.nii.gz

bl login

bl pub query -q o3d --json | jq .[0].releases
[
  {
    "removed": false,
    "_id": "5bab993aa918ae0027024192",
    "name": "1",
    "create_date": "2018-09-01T00:00:00.000Z"
  },
  {
    "removed": true,
    "_id": "5bb2437c19fdb40027307f7b",
    "name": "",
    "create_date": "2018-10-01T00:00:00.000Z"
  },
  {
    "removed": false,
    "_id": "5c0ff604391ed50032b634d1",
    "name": "2",
    "create_date": "2018-12-11T00:00:00.000Z"
  }
]

hi @arokem see here: https://brainlife.io/docs/cli/download/
We require login. This is because the data, well contains brains! :-) what is the issue with login?
We can create a project, add our users and do one fo the following
See examples:
$ bl data download -i 5a050ad5d76a2a002737e572
Download dataset to 5a050ad5d76a2a002737e572
100% done

$ ls 5a050ad5d76a2a002737e572
t1.nii.gz

bl login

bl pub query -q o3d --json | jq .[0].releases
[
  {
    "removed": false,
    "_id": "5bab993aa918ae0027024192",
    "name": "1",
    "create_date": "2018-09-01T00:00:00.000Z"
  },
  {
    "removed": true,
    "_id": "5bb2437c19fdb40027307f7b",
    "name": "",
    "create_date": "2018-10-01T00:00:00.000Z"
  },
  {
    "removed": false,
    "_id": "5c0ff604391ed50032b634d1",
    "name": "2",
    "create_date": "2018-12-11T00:00:00.000Z"
  }
]

Yeah. Fair enough.
@francopestilli : is there a way to create a dataset that don't require authentication on brainlife? Otherwise, I think that we can also use zenodo or figshare for this data.

The message below
https://github.com/tee-ar-ex/trx-python/pull/32#issuecomment-1138507901

hi @arokem see ehre: https://brainlife.io/docs/cli/download/
We require login. This is because the data, well contains brains! :-) what is the issue with login?
We can create a project, add our users and do one fo the following
See examples:
$ bl data download -i 5a050ad5d76a2a002737e572
Download dataset to 5a050ad5d76a2a002737e572
100% done

$ ls 5a050ad5d76a2a002737e572
t1.nii.gz

bl login

bl pub query -q o3d --json | jq .[0].releases
[
  {
    "removed": false,
    "_id": "5bab993aa918ae0027024192",
    "name": "1",
    "create_date": "2018-09-01T00:00:00.000Z"
  },
  {
    "removed": true,
    "_id": "5bb2437c19fdb40027307f7b",
    "name": "",
    "create_date": "2018-10-01T00:00:00.000Z"
  },
  {
    "removed": false,
    "_id": "5c0ff604391ed50032b634d1",
    "name": "2",
    "create_date": "2018-12-11T00:00:00.000Z"
  }
]

The issue with authentication is that it puts a burden on potential developers and users of the software that have no interest in brainlife per se to get an account on brainlife in order to run the standard tests of the software. For example, getting the software from GitHub does not require a GitHub account. Getting data from figshare requires no figshare account. I understand that for many cases you need to protect the data on brainlife, by requiring an account and authentication, but arguably this is not one of these cases.

Figshare is good for me, I can upload it tomorrow. Regarding Brainlife, one solution to be developed could be to allow a project to be marked as public by its owner to allow open access.

@anibalsolon Would it be possible to (once on Figshare) to do a PR to replace the fetcher for the tests?

For sure!

Just checking in again: is this still WIP? What's the remaining TODOs here? Just the docstrings?

Sorry I was moving and now OHBM, I will finish that this week hopefully !


[yasm-tool] build failure ( all ports failing )

Host Environment

OS: Windows 10
Compiler: visual studio 2019 16.10.1

To Reproduce
Steps to reproduce the behavior:
.\vcpkg --triplet x86-windows install yasm-tool
Failure logs
Detecting compiler hash for triplet x64-windows...
Detecting compiler hash for triplet x86-windows...
Could not locate cached archive: C:\Users\brody\AppData\Local\vcpkg\archives\c6\c6e03165c846e61635df4879070ed42c0344d0aa.zip
Could not locate cached archive: C:\Users\brody\AppData\Local\vcpkg\archives\7c\7cc6964bbf071bad29ab38b4433b787121457832.zip
Could not locate cached archive: C:\Users\brody\AppData\Local\vcpkg\archives\e5\e523010c0e48b0cc7cc11345625d3e820a90a125.zip
Starting package 1/3: yasm-tool:x64-windows
Building package yasm-tool[core]:x64-windows...
-- Using C:/Development/vcpkg/downloads/yasm-yasm-009450c7ad4d425fa5a10ac4bd6efbd25248d823.tar.gz
-- Cleaning sources at C:/Development/vcpkg/buildtrees/yasm-tool/src/d25248d823-818ed1bbf4.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/Development/vcpkg/downloads/yasm-yasm-009450c7ad4d425fa5a10ac4bd6efbd25248d823.tar.gz
-- Using source at C:/Development/vcpkg/buildtrees/yasm-tool/src/d25248d823-818ed1bbf4.clean
-- Installing: C:/Development/vcpkg/packages/yasm-tool_x64-windows/share/yasm-tool/copyright
-- Configuring x64-windows
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:105 (message):
Command failed: ninja -v
Working Directory: C:/Development/vcpkg/buildtrees/yasm-tool/x64-windows-rel/vcpkg-parallel-configure
Error code: 1
See logs for more information:
C:\Development\vcpkg\buildtrees\yasm-tool\config-x64-windows-out.log
Call Stack (most recent call first):
scripts/cmake/vcpkg_configure_cmake.cmake:323 (vcpkg_execute_required_process)
ports/yasm-tool/portfile.cmake:26 (vcpkg_configure_cmake)
scripts/ports.cmake:139 (include)
Error: Building package yasm-tool:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: yasm-tool:x64-windows
Vcpkg version: 2021-05-05-9f849c4c43e50d1b16186ae76681c27b0c1be9d9
config-x64-windows-out.log
CMakeOutput.log
CMakeError.log
Additional context
This error is happening with every port I try and install, I'm sure it is something wrong on my side but I have no idea how to fix this. I have tried updating and reinstalling vcpkg and visual studio but no luck yet. Any help would be appreciated.

@YoungBlood01, it looks something wrong in windows sdk, it could not find the mt, could you reinstall/repair windows sdk and try again?
 cmd.exe /C "cd . && C:\Development\vcpkg\downloads\tools\cmake-3.20.2-windows\cmake-3.20.2-windows-i386\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\cmTC_ba187.dir --rc=rc --mt=CMAKE_MT-NOTFOUND --manifests  -- C:\PROGRA~2\MIB055~1\2019\COMMUN~1\VC\Tools\MSVC\1429~1.300\bin\Hostx64\x64\link.exe /nologo CMakeFiles\cmTC_ba187.dir\testCCompiler.c.obj  /out:cmTC_ba187.exe /implib:cmTC_ba187.lib /pdb:cmTC_ba187.pdb /version:0.0 /machine:x64  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
    RC Pass 1: command "rc /fo CMakeFiles\cmTC_ba187.dir/manifest.res CMakeFiles\cmTC_ba187.dir/manifest.rc" failed (exit code 0) with the following output:
    The system cannot find the file specified
    ninja: build stopped: subcommand failed.

@YoungBlood01, does this still be a problem for you?

@YoungBlood01, please reopen this issue if it still be a problem for you.


Speakers Page

Is your feature request related to a problem? Please describe.
Implement a speakers page as shown in the design below.
Describe the solution you'd like

Create a reusable <SpeakerCard /> component
Use a FlatList to render a list of all speakers

Describe alternatives you've considered
Additional context

@cindykandie i will work on this


Fish key bindings do not support 'become(vim)' action

[x] I have read through the manual page (man fzf)
[x] I have the latest version of fzf
[x] I have searched through the existing issues

Info

OS

[ ] Linux
[x] Mac OS X
[ ] Windows
[ ] Etc.

Shell

[ ] bash
[ ] zsh
[x] fish

Problem / Steps to reproduce
Fish key bindings do not support become(vim {}) action. This is because vim cannot be piped to some other command, and if you take a look at key bindings, this is exactly the case:

https://github.com/junegunn/fzf/blob/d7b61ede07ab20ba106191968f12aaf600b6ede4/shell/key-bindings.fish#L39

I am sure if there is a way around this to keep the same functionality. I am opening this issue to discuss possibilities to improve key bindings to support this.
Workaround
I personally edited the fzf-file-widget function so that it does not support existing input on the command line, but instead it simply launches fzf with empty query string and then you can enter your query. For me this is sufficient since it aligns well with how I work, and it is more important to me to be able to switch into vim from fzf.
diff --git a/shell/key-bindings.fish b/shell/key-bindings.fish
index 69f7697..74c708c 100644
--- a/shell/key-bindings.fish
+++ b/shell/key-bindings.fish
@@ -36,20 +36,9 @@ function fzf_key_bindings
     test -n "$FZF_TMUX_HEIGHT"; or set FZF_TMUX_HEIGHT 40%
     begin
       set -lx FZF_DEFAULT_OPTS "--height $FZF_TMUX_HEIGHT --reverse --scheme=path --bind=ctrl-z:ignore $FZF_DEFAULT_OPTS $FZF_CTRL_T_OPTS"
-      eval "$FZF_CTRL_T_COMMAND | "(__fzfcmd)' -m --query "'$fzf_query'"' | while read -l r; set result $result $r; end
-    end
-    if [ -z "$result" ]
-      commandline -f repaint
-      return
-    else
-      # Remove last token from commandline.
-      commandline -t ""
-    end
-    for i in $result
-      commandline -it -- $prefix
-      commandline -it -- (string escape $i)
-      commandline -it -- ' '
+      eval "$FZF_CTRL_T_COMMAND | "(__fzfcmd)' --multi'
     end
+    commandline -t ""
     commandline -f repaint
   end

I don't think this is a problem. CTRL-T is used to complete the command line.
$ vim <CTRL-T>

The calling script expects fzf to print the file path so that it can paste the result onto the command line. If you turn fzf into something else that doesn't do what it's supposed to do, the script will surely break no matter what.
It's not clear where the become action you described is coming from, but if you have it in your $FZF_DEFAULT_OPTS, I'd say that's not a good idea because fzf is a general-purpose text filter that can be used with any kind of text that will not work with vim {}. e.g. seq 1000 | fzf

Thanks. Sorry for confusion, I misused CTRL-T command. I didn't have it in my FZF_DEFAULT_OPTS, but in  FZF_CTRL_T_OPTS. However, I removed it now, and defined a separate binding which allows me to open different kinds of files.