Missing documentation of how to customize Custom Field Services

This document is pretty good overall, very detailed and complete. The one thing my team found while working with it was that there is no documentation on how to customize CFS. We have created some internal doc that shows how to customize some parts of CFS. Let me know what you think.
Thanks!
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c7f3d28a-1028-af04-4f8c-656dd2ea0fbb
Version Independent ID: 5dc125ab-7859-9b2b-ca4f-153aea32b639
Content: Build your own Connected Field Service (IoT) solution
Content Source: ce/field-service/connected-field-service-overview.md
Service: dynamics-365-customerservice
GitHub Login: @shellyhaverkamp
Microsoft Alias: shellyha

Thanks for reaching out, appreciate the help, I have reached out to you via Teams.


Feature Request: support for LaTex with mathjax

Now AnkiTab cannot display cards contains latex formals.
Wish support it in near future.
Thank you!

Sorry! I found AnkiTab can display LaTex, only the mathjax format doesn't support.

If you send me some cards that include MathJax cards for testing I can include MathJax functionality into ankiTab (should be pretty easy as it's only including more scripts).

Sorry! I've already changed all the cards to LaTex. However, I found AnkiTab doesn't support LaTex shorthand.

Hi, @corollari I'd love to see this feature implemented (MathJax).
I can provide you with example cards, but it's rather simple - just create a card with, say:
\( f(x) = x^2 \)
MathJax is rendered live and higher resolution than LaTeX, plus my workflow allows me to use only MathJax.

@masteranza Noted. I'll keep it here till I have time to work on it.

same here, I cannot really use the plugin without it.

Just added MathJax support, it's currently working on branch mathjax. Not going to push the next version live yet because:

Firefox: I have to make sure that all the new files added comply with Firefox policies (including tracking down the all the dependencies of MathJax in it's unminimized form, checking if all the versions are a "stable release", checking licenses...)
Chrome: I have tested this version with my usual set of testing cards but i'm not 100% sure that I'm not adding a new edge case that causes some special type of cards to crash as I haven't been able to review all of the code on the new mathjax files. If someone is willing to test it and reports that it's working I'll push it live.

Hope it's worth the wait @Novite5 and @masteranza. If there's anything else just feel free to create a new issue :)

@corollari I just tested it, works fine for my cards (simple basic model). Thanks a lot!

@corollari It works great! Thank you for making this happen!

@Novite5 @annargrs Thanks, I'll push it live later today!

Published! Should be available in the chrome store in a few hours.

Great work! Thank you!


Parameters are not passed due to TypeError in msct_parser

Prerequisites

[x] Did you check past issues for similar issue(s) ?
[x] Can you reproduce the issue with any of the files found in sct_testing_data ?
[x] If you are using a development environment, is your environment updated ?

Description
The flag -param does not parse and pass the arguments properly (i.e. it has no effect).
Steps to Reproduce
sct_label_vertebrae -i t2r.nii.gz -s t2r_seg.nz -c t2 -qc ${PATH_QC} -v 2 -r 0 -param size_IS_initc2=41,size_AP_initc2=25,shift_AP_initc2=50

--
Spinal Cord Toolbox (master/caf11f46d92fe8765490a53325acbf0e2b36b8f1)
Running /Users/julien/code/sct/scripts/sct_label_vertebrae.py -i t2r.nii.gz -s t2r_seg.nii.gz -c t2 -qc /Users/julien/Dropbox/documents/processing/20180424_nadia/quality_control/ -v 2 -r 0 -param size_IS_initc2=41,size_AP_initc2=25,shift_AP_initc2=50
Traceback (most recent call last):
  File "/Users/julien/code/sct/python/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
  File "/Users/julien/code/sct/python/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
  File "/Users/julien/code/sct/python/lib/python2.7/logging/__init__.py", line 465, in format
    record.message = record.getMessage()
  File "/Users/julien/code/sct/python/lib/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file msct_parser.py, line 251

The parser is working well and the expected results are obtained. I think there was an error typing the segmentation file. does not have the correct extension should be nii.gz instead of nz.
PS: if I put the -qc option I get errors related to QC. I have the dependencies updated.
QC: layout with Sagittal slice Traceback (most recent call last):   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/scripts/sct_label_vertebrae.py", line 941, in <module>     main()   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/scripts/sct_label_vertebrae.py", line 356, in main     generate_qc(fname_in, labeled_seg_file, args, path_qc)   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/scripts/sct_label_vertebrae.py", line 409, in generate_qc     qcslice_layout=lambda x: x.single(),   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/python/lib/python2.7/site-packages/spinalcordtoolbox/reports/qc.py", line 401, in add_entry     layout(qcslice)   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/python/lib/python2.7/site-packages/spinalcordtoolbox/reports/qc.py", line 169, in wrapped_f     img, mask = func(sct_slice, *args)   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/python/lib/python2.7/site-packages/spinalcordtoolbox/reports/qc.py", line 399, in layout     return qcslice_layout(qslice)   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/scripts/sct_label_vertebrae.py", line 409, in <lambda>     qcslice_layout=lambda x: x.single(),   File "/Users/fperdigon_admin/Documents/spinalcordtoolbox/python/lib/python2.7/site-packages/spinalcordtoolbox/reports/slice.py", line 300, in single     assert len(set([x.data.shape for x in self._images])) == 1, "Volumes don't have the same size" AssertionError: Volumes don't have the same size

thank you @fperdigon, well spotted. However the output verbose is not acceptable: it should say something like "this file does not exist" or something like that... could you appropriately rename this issue and address it?
regarding the QC issue: pls open a new issue with instructions to reproduce

@jcohenadad  This issue was fixed by @zougloub here. I already tested and the parser is working fine. I will close this issue.


[devil] build failure

Host Environment

OS: Windows 10 x64
Compiler: gcc 10.2.0 mingw64

To Reproduce
Steps to reproduce the behavior:

./vcpkg install devil
Try to build a project with ILUT
Somewhere in the C file:

#include <IL/il.h>
#include <IL/ilut.h>

Somewhere in the CMake file:
find_package(DevIL REQUIRED)
target_link_libraries(main PRIVATE ${IL_LIBRARIES} PRIVATE ${ILUT_LIBRARIES})

Failure logs
PS S:\c\C\compute_test\build> make
[ 50%] Building C object CMakeFiles/compute_test.dir/main.c.obj
In file included from S:\c\C\compute_test\main.c:11:
C:/src/vcpkg/installed/x64-windows/include/IL/ilut.h:100:11: fatal error: IL/ilut_config.h: No such file or directory
  100 |  #include <IL/ilut_config.h>
      |           ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles\compute_test.dir\build.make:76: CMakeFiles/compute_test.dir/main.c.obj] Error 1
make[1]: *** [CMakeFiles\Makefile2:82: CMakeFiles/compute_test.dir/all] Error 2
make: *** [Makefile:90: all] Error 2

Additional context
This issue goes away when I copy ilut_config.h from DevIL/include/IL into vcpkg/installed/x64-windows/include/IL.
It seems like upon installing DevIL, this file fails to be copied.

@chaquator, thanks for reporting this issue!
It looks the headers hasn't been installed, see https://github.com/DentonW/DevIL/blob/master/DevIL/src-ILUT/CMakeLists.txt#L102, could you also report an issue to Upstream?

@chaquator, I have submitted a PR to fix this, btw, I have already report this issue to Upstream https://github.com/DentonW/DevIL/issues/96

Awesome, thank you so much!


openssl dgst -r breaks dangerously on filenames with newlines (does not emulate coreutils behavior)

When openssl dgst -r encounters a filename that has a newline in it, it can be convinced to emit an uninterpretable digest summary:
0 dkg@alice:/tmp/cdtemp.6F5CpY$ for x in a b 'a b' "$(printf 'a\nb')"; do echo "$x" > "$x"; done
0 dkg@alice:/tmp/cdtemp.6F5CpY$ sha256sum *
87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7  a
\911169ddaaf146aff539f58c26c489af3b892dff0fe283c1c264c65ae5aa59a2  a\nb
01186fcf04b4b447f393e552964c08c7b419c1ad7a25c342a0b631b1967d3a27  a b
0263829989b6fd954f72baaf2fc64bc2e2f01d692d4de72986ea808f6e99813f  b
0 dkg@alice:/tmp/cdtemp.6F5CpY$ openssl dgst -r *
87428fc522803d31065e7bce3cf03fe475096631e5e07bbd7a0fde60c4cf25c7 *a
911169ddaaf146aff539f58c26c489af3b892dff0fe283c1c264c65ae5aa59a2 *a
b
01186fcf04b4b447f393e552964c08c7b419c1ad7a25c342a0b631b1967d3a27 *a b
0263829989b6fd954f72baaf2fc64bc2e2f01d692d4de72986ea808f6e99813f *b
0 dkg@alice:/tmp/cdtemp.6F5CpY$

If this is used in an automated way to create digests of files whose names are manipulable by a malicious party, that party could inject spurious digests in the output.
coreutils describes the escaping rules in the md5sum documentation.  openssl dgst -r should follow suit:

if [the file name] contains a backslash or newline, the line is started with a backslash, and each problematic character in the file name is escaped with a backslash, making the output unambiguous even in the presence of arbitrary file names.

Interesting.  I am also able to reproduce this too.
So, following output from sha256sum * would be similar for openssl dgst -r *:
\911169ddaaf146aff539f58c26c489af3b892dff0fe283c1c264c65ae5aa59a2  a\nb

# instead of:

911169ddaaf146aff539f58c26c489af3b892dff0fe283c1c264c65ae5aa59a2 *a
b

@dkg A fix for this issue has now gone into master as of last night.
https://github.com/openssl/openssl/pull/8578

Ah, right, so let's close this


Getting "PR is already touching this file" warning even though PR is closed

On a file in the PreMiD/Schemas repository (specifically https://github.com/PreMiD/Schemas/blob/master/schemas/metadata/1.0.json) Refined GitHub thinks that an already merged PR is still open and always warns me that it's touching the file. However, said PR is merged: PreMiD/Schemas#1.
When PreMiD/Schemas#2 was created and closed normally, however, the warning disappeared saying that multiple PRs were touching the file.

The PR was merged recently. What you’re seeing is just cache that hasn’t expired yet

I just saw this today for a PR that closed in Feb ... how long does the cache last?

It should only last 1 + 9 days: https://github.com/sindresorhus/refined-github/blob/2235c23dedef9f5ef91cc07c53175e28e5f64b6d/source/features/list-prs-for-file.tsx#L91-L92 (meaning that if you open it after 24 hours of the closure, it will revalidate the cache, so if you refresh the page it should be gone)
Is is still there for you now? I never see it for very long

I already opened the PR and it's since been merged so it's impossible to test again with this scenario. If I see it again, I'll try refreshing the page.


cu3 image is 71 days old?

When using a cu3 image I get a warning that the image is 71 days old, but cu3 came out closer to 40 days ago.
It's not a big problem I can just set accept_outdated=y if I need to keep it. But I wonder where this date comes from.
Using image mcr.microsoft.com/businesscentral/onprem:cu3-se-ltsc2016
Country se
Version 13.3.27233.0
Locale sv-SE
Running mcr.microsoft.com/businesscentral/onprem:cu3-se-ltsc2016 (this will take a few minutes)
NavContainerHelper is version <IP_ADDRESS>
Host is Microsoft Windows Server 2016 Datacenter - ltsc2016
Docker Client Version is 17.06.2-ee-7
Docker Server Version is 17.06.2-ee-7
Pulling image mcr.microsoft.com/businesscentral/onprem:cu3-se-ltsc2016
cu3-se-ltsc2016: Pulling from businesscentral/onprem
Using image mcr.microsoft.com/businesscentral/onprem:cu3-se-ltsc2016
-- removed lines --
WARNING: You are running a container which is 71 days old.
Microsoft recommends that you always run the latest version of our containers.

71 days is actually the age of the generic image (the foundation of the specific image)
I will create a better message and/or way to measure it.


Add convenience option function to set custom logger

This adds a convenience function, same to SetCodec etc to set an alternative logger, such as logrus etc.
Currently it is possible this way:
path = "/tmp/foo"
options := badger.DefaultOptions(path)
options.Logger = logrus.New()
db, err = bow.Open(path, bow.SetBadgerOptions(options))
...

But also could be possible just this, if the case is to only swap quite verbose Badger's logging:
path = "/tmp/foo"
db, err := bow.Open(path, bow.SetLogger(logrus.New()))

That's neat. Thanks for contributing!


Merge editor looses conflict resolution lense actions

After I did "Accept Combination" on the first conflict (I picked the left hand side), all actions are gone for resolving each conflict:

state.json.zip

Maybe this log entry:
ERR Assertion Failed: Error: Assertion Failed
    at D (vscode-file://vscode-app/Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:11:9896)
    at new r (vscode-file://vscode-app/Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1041:10074)
    at vscode-file://vscode-app/Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1041:13755
    at Array.map (<anonymous>)
    at e.MergeDiffComputer.computeDiff (vscode-file://vscode-app/Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1041:13748)

There is also a git exception it seems:

Reproduces running out of sources, stack:
assertFn (assert.js:45)
DetailedLineRangeMapping (mapping.js:153)
(anonymous) (diffComputer.js:37)
computeDiff (diffComputer.js:37)
await in computeDiff (async)
recompute (textModelDiffs.js:58)
(anonymous) (textModelDiffs.js:27)
_runIfNeeded (autorun.js:76)
AutorunObserver (autorun.js:55)
autorun (autorun.js:10)
TextModelDiffs (textModelDiffs.js:25)
MergeEditorModel (mergeEditorModel.js:32)
_createInstance (instantiationService.js:91)
createInstance (instantiationService.js:62)
createInputModel (mergeEditorInputModel.js:210)
await in createInputModel (async)
resolve (mergeEditorInput.js:72)
setInput (mergeEditor.js:121)
await in setInput (async)
doSetInput (editorPanes.js:292)
doOpenEditor (editorPanes.js:166)
openEditor (editorPanes.js:65)
(anonymous) (editorGroupView.js:809)
doShowEditor (editorGroupView.js:829)
doOpenEditor (editorGroupView.js:794)
openEditor (editorGroupView.js:711)
openEditor (editorService.js:408)
await in openEditor (async)
run (commands.js:63)
handler (actions.js:358)
invokeFunction (instantiationService.js:46)
_tryExecuteCommand (commandService.js:81)
executeCommand (commandService.js:50)
$executeCommand (mainThreadCommands.js:75)
_doInvokeHandler (rpcProtocol.js:381)
_invokeHandler (rpcProtocol.js:366)
_receiveRequest (rpcProtocol.js:307)
_receiveOneMessage (rpcProtocol.js:234)
(anonymous) (rpcProtocol.js:115)
invoke (event.js:660)
deliver (event.js:836)
fire (event.js:800)
fire (ipc.net.js:504)
port.onmessage (localProcessExtensionHost.js:452)

This seems to work now (did some bugfix for a different issue).


reopen: "Duplicate rows added on window resize"

(reopening, since it's not clear whether you noticed additional commentary on #98 after it was closed)
I'm afraid the commit doesn't quite address the issue effectively -- the very next call to update(), either before or after switching back from another tab, causes the same issue to occur.
http://codepen.io/anon/pen/PpwBGV
... added an update() timer at the bottom to emulate the behavior

@yarf Hi, did you have a chance to test this? Does it works as expected?

yup, that did it!
Do we now get a free commercial-license for being part of your QA team? ;-)
cough

I'm glad it helped!
Uplabs is responsible for licenses, not me :P


How do I start it?

Describe the bug

Install .NET SDK 6.0
Install the files into your root directory.

Screenshots

Additional context
You can contact me on discord at SamzyDev#1205.

You can use the below command to start it. Of course, you'll need to get the .net 6 runtime installed.
dotnet Blogifier.dll

I create a post on my own Blogifier site on How to install Blogifier on AWS using the free tier. If you haven't gotten it working, check it out and let me know.
https://blog.davidrandoll.com/posts/host-blogifier-for-free-on-aws

@david-randoll Can you please try to access the blog post that you just mentioned here? I think your blog site not responding now. Although I can run this application successfully. I just wanted to read the post.

@david-randoll Can you please try to access the blog post that you just mentioned here? I think your blog site not responding now. Although I can run this application successfully. I just wanted to read the post.

See if this works.
https://link.medium.com/7Di17bwKMAb

Yeah, It's working. Thanks


Cleaning up courses template

Does this issue mean fix up this file https://github.com/PittCSWiki/pittcswiki.github.io/blob/master/helpers/courses/generate_page_skeletons.js

It's basically fixing design and getting data more data into the course pages

Just the start of a new component for testimonials dbe5045 . Though the data column names need to be modified cause lol really termSzn

Thanks - and that component/idea is still in an experimental phase


Access to file system from lsp server

Hi,
Not having vscode.workspace.fs on the server side of an extension poses quite a challenge.
One way I see to overcome this is the way anycode does it:
Add a handler to the client for fetching file contents:
client.onRequest('persisted/read', async (): Promise<string> => {
  if (!persistUri) {
     return '';
  }
  try {
     const data = await vscode.workspace.fs.readFile(persistUri);
     return decoder.decode(data);
  } catch {
     return '';
  }
});

And send a request from the server whenever you need to read files:
const raw = await this._connection.sendRequest<string>('persisted/read');

But I noticed that anycode only does this in the node runtime. Does it mean that this won’t work in the browser?
Is there a more direct way to allow the server access to the file system or will there be one in the future?
I’m asking because I’m working on migrating the Wake lsp extension to the browser. Its server is compiled from C++ to wasm and it requires access to the file system, so I'm struggling to make the transition.

Dups https://github.com/microsoft/vscode-languageserver-node/pull/782.


VM Exception While Processing Revert.

Hello, I am having trouble replicating your work, which overall is great.
I am using Truffle5 and Solc 0.4.23;
I am following along using a basic uint256 operations contract:
`
pragma solidity ^0.4.23;
import "../upgradeable/upgradeable.sol";
contract CalculatorV5 is Upgradeable {
uint256 internal _count;
constructor(uint256 count)public {
    _count = count;
}

function increment(uint256 amount)
    public
    returns (bool)
{
    require(amount > 0, "amount must be positive");
    _count = _count + amount;
    return true;
}

function getCount()
    external
    view
    returns(uint256)
{
    return _count;
}

}
For test using a .js type of tests (vs .sol):
const UpgradeableProxy = artifacts.require('./upgradeable.sol')
const CalculatorV5 = artifacts.require('./CalculatorV5.sol');
const CalculatorV6 = artifacts.require('./CalculatorV6.sol')
contract('Calculator', (accounts) => {
    beforeEach(async () => {
        calcV5 = await CalculatorV5.new(100);
        uproxy = await UpgradeableProxy.new(calcV5.address);
        calcV5Proxy =  await CalculatorV5.at(uproxy.address);
        await calcV5Proxy.initialize();
    });
   it('calc v5 should equal 100',async () => {
        let value = await calcV5Proxy.getCount({from:accounts[1]});
       // also tried  let value = await calcV5Proxy.getCount();
     // as well as  let value = await calcV5Proxy.getCount.call();
        assert.equal(value, 100);
    });

When I run it the before each works perfectly, emits the event Initalized, but fails on the test with this vague response:
Contract: Calculator
calc v5 should equal 100:
Error: Returned error: VM Exception while processing transaction: revert
Please let me know if you need more information. Thanks!

Edit: Minor grammar and styling errors.


no definition for "arm_cfft_init_1024_f32"

Apologies for a simple question here, I've followed the usage instructions and explored all option to resolve the issue but with no luck.
We are are having trouble in compiling the arm_fft_bin_example.ewp. There is linker error for the arm_cfft_init_1024_f32 function.
I have added below library -
C:\Program Files\IAR Systems\Embedded Workbench 9.2\arm\CMSIS\Lib\IAR\iar_cortexM4lf_math.a
I have added below path for header files -
..\IAR-CMSIS-DSP\CMSIS_5\CMSIS\Core\Include
..\IAR-CMSIS-DSP\CMSIS-DSP\Include
..\IAR-CMSIS-DSP\CMSIS-DSP\Include\dsp
Development environment

Error displayed -
Error[Li005]: no definition for "arm_cfft_init_1024_f32" [referenced from ..\IAR-CMSIS-DSP\Examples\cortexM4lf\Obj\arm_fft_bin_example\arm_fft_bin_example_f32.o]
Are we missing any additional settings.
Reference - https://github.com/ARM-software/CMSIS_5/issues/1670
Additionally can anyone please confirm example is for 1024 samples but testInput_f32_10khz (test input data) is of 2048, it is always expected the data samples to be double of processing samples??

@vipinthomas123 Can you tell me which version of CMSIS-DSP is provided with your IAR environment ?

I believe it's CMSIS 5.7
One update
If I use arm_cfft_init_f32(&varInstCfftF32,1024), I am able to compile without any error.

@christophe0606 can you pls confirm this same CMSIS library can be used for Cortex R4? Or if there is different repo, pls share the link. Thank you

@vipinthomas123 You're using an older version. So you can initialize with arm_cfft_init_f32 but it may not be good for code size since it prevents the linker from deducing which FFT lengths you really use.
You may use the old way of initializing by settings the field of arm_cfft_instance_f32 using structures from CommonTables/arm_const_structs_f32.c. For instance, arm_cfft_sR_f32_len512 for length 512. In that case, you don't use arm_cfft_init_f32.
Note that this method of initializing will not work with Helium instruction set.
The library is never tested on Cortex-R because we focus on Cortex-M and have recently started to look at Cortex-A.
It should work on Cortex-R4 if it is supported by CMSIS-Core. If it is not supported by CMSIS-Core, you won't be able to build and you should open a github issue on the CMSIS-6 repository to ask for R4 support.


Add full location path to Stock Item location entry

Can this feature also be added for the Stock Item view?
Right now I have individual locations set up for each section of a tray, where the path is something like Tray X/A4.
If I have multiple identical trays the Stock Item view only shows the last part of the path A4, which is ambiguous:

Originally posted by @Gigahawk in https://github.com/inventree/InvenTree/issues/3685#issuecomment-1256045401

There's a fully-qualified path here:

Is this not sufficient?

I suppose, just seems like some wasted there, especially since the short strings code is already implemented elsewhere

The "short strings" is implemented in a different spot and won't transfer to rendering the data here. As the fully qualified path is already displayed here, I'm going to close this one out :)


想問一下變形校正問題

你好
能否單純做一個變形校正的代碼
我已經有一大堆用yolo裁切車牌的小圖片
車牌顏色有白底 紅底 綠底跟黃底
但因為傾斜車牌 很難校正 不知道你那邊方不方便釋出旋轉及校正的代碼
謝謝

你好 能否單純做一個變形校正的代碼 我已經有一大堆用yolo裁切車牌的小圖片 車牌顏色有白底 紅底 綠底跟黃底
但因為傾斜車牌 很難校正 不知道你那邊方不方便釋出旋轉及校正的代碼 謝謝

Hi,
thanks for your attention. For the rectification, this project adopts two different methods, named "Shape-based Rectification" and "Color-based Rectification".
The corresponding codes are the functions ChineseLicensePlateDetector.shape_rectification and ChineseLicensePlateDetector.color_rectification. You can see the pipeline diagrams in README/structures to understand their operations. Roughly speaking, they both receive the RoI which is the "small cut" of the region of the plate, and then calculate 4 corner points of the slanted plate to calculate the perspective transformation matrix, and then apply to the RoI to output the rectified small image. And these operations depend on some pre-defined hyper-parameters, which can be found in config and may need tuning to specific data.
Hope that helps!


DOCS: 4.x REST Client

Problem Description
We need to update docs for 4.x

[x] update restclient.adoc for MP

Guidelines

[x] Make sure any fixes from SE 3.x are forward ported to 4.x #7515
[x] Make sure any fixes from MP 3.x are forward ported to 4.x
[x] Initial focus is on correctness
[x] Record migration impact in this issue

Migration Impact
Summarize incompatibilities between Helidon 3 and Helidon 4.
Summarize actions customers must take to migrate code
OK to reference external specs or documents (for example MicroProfile specs)

There were no changes impacting user in the rest client. I went though the docs and everything seems correct. Also no special migration is required since there are no changes.


operator uma-admin-operator (2024.10.2)

/test skip check_pruned_graph

@mn660971: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

/ok-to-test

/ok-to-test


WIP add Request::initialize_as_target

I took a stab at adding a initialize_as_target Request builder.
I'm afraid I currently don't know enough about this kind of functionality so I'll leave some questions here.
@averyanalex maybe you could shed some light on them?

What I can say about default values: with everything set to [0; N], TargetMode::PassiveOnly and TargetSelRes::PICC, my Android phone detects FeliCa:

When I set params to some random recommendations from stackoverflow, NFC Tools can't see pn532, but it responses to microcontroller. (so there is a connection, but chip is not recognized by phone).
Also I don't know how to emulate ISO 14443-3A (but I want to do it).

Just thought I'd let you know. I'm rather limited on time currently so it might take me some weeks to properly finish this.
(btw, I spent like 4 hours today on https://github.com/esp-rs/esp-hal/issues/295#issuecomment-1410885751 which you also seem to have encountered 🤣)

Being honest, I don't think I'll find the time to finish this.
If anyone's interested they can continue from here :)


Error 403 after login

I'm using the 'profile' has scope but when i loggin the url contains :
"code=...&scope=https://www.googleapis.com/auth/userinfo.profile"
https://www.googleapis.com/auth/userinfo.profile is outdated (i read in docs) , because of this scope the page gives a error 403 Forbidden page acess. If i remove the 'scope=https://www.googleapis.com/auth/userinfo.profile" part on URL everythings seems ok.
Thanks

I'm having a hard time understanding what issue you experiencing. Could you attach a snippet of code that is not working and explain what behavior you are expecting to see.

If i remove the scope parameter the login and redirect happens with success

Check out the latest version of this example in this repo.
Please let me know where you got that older version from and I will try to update it. Thanks

Thanks Matt, but i already used the newest version (install via composer and directly), same results.
I used the example idtoken.php in this repo, but gives me every time 403 Forbidden, the 'profile' scope isnt working.

Thanks Matt, but i already used the newest version (install via composer and directly), same results.
I used the example idtoken.php in this repo, but gives me every time 403 Forbidden, the 'profile' scope isnt working.

I seem to have avoided answering your question directly, sorry about that, that was unintentional.  I would double check your script against the version I linked and make sure there aren't any other changes you need to make.

Let me know if you have any other questions.

4youplay
Have you find answer ? I am facing same problem.

If your public application uses scopes that permit access to certain user data, it must pass review. If you see unverified app on the screen when testing your application, you must submit a verification request to remove it. Find out more about unverified apps https://support.google.com/cloud/answer/7454865 in the Help Center. I had to go through verification too, just to use "scope" => "profile email" ps: the reason of 403 is stated in the error itself s pozdravem, Filip Oščádal https://fredbrooker.gscloud.cz https://fredbrooker.gscloud.cz
…
On Wed, Dec 26, 2018 at 7:34 PM sharmasandeep410 @.*> wrote: 4youplay Have you find answer ? I am facing same problem. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1527 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEmXrkUmD0Ot_0bk7XSxS-KwZtD6-DUks5u88E3gaJpZM4W5BqD .

I also thought so: using profile scope will need my app to be verified by Google. But I don't see any details in the error, just the 403 error.


ERROR: Could not resolve "@badeball/cypress-cucumber-preprocessor"

Current behavior
I am trying to run tests with cypress and cucumber, but, after installing and configuring. I always get this.

I can't really tell whats happening, I have also tried to update my node to the latest stable version, but still failing.
Desired behavior

Test code to reproduce
The github repo is below
https://github.com/Jexsie/cypressCucumber
Versions

Cypress version:  11.0.1
Preprocessor version: 14.0.0
Node version: 18.12.1

Checklist

[x] I've read the FAQ.
[x] I've read Instructions for logging issues.
[x] I'm not using<EMAIL_ADDRESS>(package name has changed and it is no longer the most recent version, see #689).

Can't reproduce. Have you actually installed the modules though?

Yes @badeball, I have installed them.  Below is my package.json
  "name": "cypress-cucumber-boilerplate",
  "version": "2.2.0",
  "description": "This project is meant to be a kind of template to start working with Cucumber and Cypress just cloning the project.",
  "main": "index.js",
  "scripts": {
    "cypress:runner": "cypress open",
    "cypress:execution": "cypress run",
    "cypress:execution-tags": "cypress run --env tags=@mobile",
    "cypress:execution-allure": "cypress run --env allure=true",
    "allure:clear": "rm -r allure-results/ allure-report cypress/screenshots || true",
    "allure:report": "allure generate allure-results --clean -o allure-report",
    "allure:history": "mv -f allure-report/history allure-results/history && rm -r allure-report || true"
  },
  "keywords": [
    "cypress"
  ],
  "author": "cucumber",
  "license": "ISC",
  "devDependencies": {
    "@badeball/cypress-cucumber-preprocessor": "^14.0.0",
    "@bahmutov/cypress-esbuild-preprocessor": "^2.1.5",
    "@shelex/cypress-allure-plugin": "^2.34.0",
    "cypress": "^11.0.1",
    "esbuild": "^0.15.13",
    "multiple-cucumber-html-reporter": "^3.0.1"
  }
}

I have tried deleting the node modules and the package.json, then reinstalling, still getting the same error. Test without cucumber, only cypress pass.
Node v18.12.1

Error: Build failed with 7 errors:
node_modules/@badeball/cypress-cucumber-preprocessor/lib/create-tests.js:6:50: ERROR: Could not resolve "@cucumber/tag-expressions"
node_modules/@badeball/cypress-cucumber-preprocessor/lib/create-tests.js:7:39: ERROR: Could not resolve "@cucumber/cucumber-expressions"
node_modules/@badeball/cypress-cucumber-preprocessor/lib/create-tests.js:8:23: ERROR: Could not resolve "uuid"
node_modules/@badeball/cypress-cucumber-preprocessor/lib/registry.js:7:39: ERROR: Could not resolve "@cucumber/cucumber-expressions"
node_modules/@badeball/cypress-cucumber-preprocessor/lib/registry.js:8:50: ERROR: Could not resolve "@cucumber/tag-expressions"

It seems @badeball/cypress-cucumber-preprocessor needs some configurations, coz, I have seen "could not resolve @cucumber/tag-expressions" The package if finely installed, and I need in configuring it, maybe. Thanks a lot @badeball for your time and support

The above error message changed when I tried to use "filterSpecs": true in my package.json

I can take care of this one @badeball.
I'm also unable to reproduce, please @Jexsie :

submit your package-lock.json.
The command you use to run Cypress

I see @Jexsie , kinda fixed then. I think we can resolve the Issue then @badeball .


Change from cli/target to <general>/target in ignore,

Remove duplicated ignore

Codecov Report

Merging #443 (17ccc16) into main (55ad4fd) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 17ccc16 differs from pull request most recent head 64b91a3. Consider uploading reports for the commit 64b91a3 to get more accurate results

@@           Coverage Diff           @@
##             main     #443   +/-   ##
=======================================
  Coverage   91.03%   91.03%
=======================================
  Files         158      158
  Lines        9239     9239
=======================================
  Hits         8411     8411
  Misses        828      828

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55ad4fd...64b91a3. Read the comment docs.


Add SPM product (library) and update dependencies

Closes #13.

Also updated travis to use latest Xcode and iOS versions.

@danqing thanks! As I am personally do not develop in Swift on daily basis could you please explain are these changes backwards compatible or not?

AFAIK this shouldn't have any compatibility issues. The only real change is that I upgraded (minor) versions of the dependencies.
The rest is just to make SPM work and update travis (testing) to latest versions of the infra.

Thanks! I will try to ask my collegues to review these changes and then merge.

@danqing looks good, thanks!


Doc strings

We need doc strings for each of these scripts.
I didn't realise that fix-gasp was basically to correct the gasp after running ttfautohint.
Cheers,
Marc

I agree we need docstrings for everything.
fix-gasp does the same thing that ttfautohint does to the gasp table, but without adding goops of hint code.

I think we would first need to have some refactoring as described at https://github.com/googlefonts/fontbakery/issues/971#issuecomment-245797089 in order to add docstring to each test function(or method).

Hey @felipesanches and @davelab6,
I would prefer FB didn't do hot fixes.
All errors should be fixed at the source imo.
My current workflow is to generate the fonts from Glyphsapp into an ofl repo structure, run FB and fix the errors in the source files until it clears FB (or as close as possible).
I've been working on a script which will fixsome of the constants for GlyphsApp sources. Its still draft.
@vmesel I understand you really wanted to tackle this issue, #971 . If the others agree that we should hot fix, then feel free to proceed. It may be better to wait until we've fixed the obvious errors due to the merge conflicts that will happen. In the meantime, I'm now testing large font families and coming across more FB related errors. I will post some up during this week. Please feel free to tackle them. They're well within your skill range and shouldn't take too long. I will provide links to all sources as well. Thank you for getting involved on this project ;)
Cheers,
Marc

hotfixes are required for projects without sources and when there is no
time/resources for recreating sources.

Yes, I agree that hotfixes are bad and that everything should be fixed in the source files. But @davelab6 has got a point here, that sometimes, in very specific situations, hotfixes are needed. So deleting them all may be throwing away useful code.

Alright, the no time constraint is a valid concern ;)

We need doc strings for each of these scripts.

This is still true :) This seems like it should be quick to do.

For the individual scripts, yes. For the main check-ttf script, no.
The problem with the check-ttf script is that it should (in my opinion) be restructured with one function per check and then (only then) each of these could have its own docstring.

@m4rc1e what do you want?

It's fine not to have a doc string for check-ttf.py. Felipe's suggestion is good. However, this is a lot of work. Better logging is more appropriate for this script imo.
I would really like them on the fix scripts, e.g fontbakery-fix-gasp.py could have.
'''
Set font gasp to 15, across all ranges. More info on can be found on the GF spec, https://github.com/googlefonts/gf-docs/blob/master/ProjectChecklist.md#gasp
'''

This pretty much summarises what the script is doing, without me having to read the code.

Merging into #681


GH-6 Version Labels

Description
Make it possible to provide a version display name on the initial version POST/PUT.
Related Issue
#6
How Has This Been Tested?

Tested locally with da-collab and da-live
Unit tests for all changes

Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[X] I have signed the Adobe Open Source CLA.
[X] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[X] I have read the CONTRIBUTING document.
[X] I have added tests to cover my changes.
[X] All new and existing tests passed.

Release notes

Ability to add a label to a version upon creation.

Additional context
Version labels provide a humane context to what a version represents. They can also be helpful for larger larger content projects. Changes happen over a series of days, weeks, or even months. This disjointed nature of content creation makes snapshot-style versions difficult to impossible to use. Being able to label versions provides a way to say in human terms, "rollback to a pre-Adobe MAX 2024 state."
DA Admin is built on top of object storage technologies. This provides a no-nonsense approach to data storage, but it lacks the granular nature of something like a NoSQL database. Object storage metadata is also immutable. What this effectively means is that if you want to update metadata, you must re-write the entire file. Moving version label creation to the POST handler ensures that we can perform one operation to create the version and label.

Thanks @auniverseaway I agree label is better so I changed it to that and also added comment.

@bosschaert I approved, but I would remove the comment part. I wouldn't build that out until someone asks for it. Label should do for the foreseeable future and we can always add comment back in later.


What exactly is "experimental" about array methods?

I really like your package and use it alot. I have never had any issues with the array methods.
However I have had issues where I had to forceInvalidations for no apparent reason. Are there edge cases where autoruns and Trackers aren't recomputated when a key in ReactiveObj change?

Array methods are experimental in the sense I'm still leaving room to change how the API is called in future versions. Particularly, I was thinking about support for Map, Set, String, other object types and dealing method name collisions. Maybe I should be more explicit about it.

However I have had issues where I had to forceInvalidations for no apparent reason. Are there edge cases where autoruns and Trackers aren't recomputated when a key in ReactiveObj change?

I'm expecting forceInvalidations only to be used when mutating values in stored in ReactiveObj directly. I'll be happy to fix edge cases that you encountered.

Thanks alot for the explanation. So far there has been just one location where i had to forceInvalidation to get the changes to reflect and the autoruns to retrigger. I have no understanding of why it is needed here, thats why i asked.
So the usecase is like this, On the click event of a button, i do a forEach loop on an array and then inside that i have another async call, in which i push a value to nested level array. I have tried doing it, by directly fetching and mutating, fetching, doing a push and then explicitly setting the key to the mutated array. The only thing that works is calling forceInvalidation. I thought it maybe related to 3 level deep function mutating, however that wasn't that case when i tried a much simpler use case on the browser console for the same key.
The code is somewhat like this (removed some extra parts):
        self.$('button.upload.images').click(function(e) {
          var uploadBtnEl = $(this),
                filesToUpload = campaignImagesDropzone.getAcceptedFiles();
          self.viewData.set('misc.uploadingImages', filesToUpload.length);
          filesToUpload.forEach(function(file) {
            file.status = Dropzone.UPLOADING;
            var fsFile = new FS.File(file);
            fsFile.owner = Meteor.userId();
            FSCampaignMedia.insert(fsFile, function(err, fileObj) {
              if (err) return campaignImagesDropzone.emit('error', file, 'failed to upload: ' + err);
              file.status = Dropzone.SUCCESS;
              file.upload.progress = 100;
              file.upload.bytesSent = file.upload.total;
              file.fsRef = fileObj;

              self.viewData.push('misc.uploadedImageIds', fileObj._id);
              self.viewData.forceInvalidate('misc.uploadedImageIds');
              self.viewData.set('misc.uploadedImages', self.viewData.get('misc.uploadedImages') + 1);
            });
          });
        });

self.viewData is a ReactiveObject here.

How is misc.uploadedImageIds accessed reactively?
Increment operation can also be written as self.viewData.update('misc.uploadedImages', function (v) { return v + 1; });

misc.uploadedImageIds is being accessed reactively through a Blaze helper:
  viewData(key) {
    if (key) return Template.instance().viewData.get(key);
    else return Template.instance().viewData.get();
  }

I didn't know i could give a function as a second parameter, will try that.

I see. I think a reproduction could be needed to fix this. Is your code open sourced?

No our source code isn't open sourced :(
I'll try and come up with a simpler reproduction so we can nail this. Thanks.


vm/vmss: expose image plan related arguments

Applicable on images with billing information. Fix #4233
Note, CLI has existing logic to retrieve such information if available in the image, so essentially these new arguments will only be useful to create a vm/vmss from custom images captured by users themselves.
Marketplace Image Plan Arguments
    --plan-name                        : Plan name.
    --plan-product                     : Plan product.
    --plan-promotion-code              : Plan promotion code.
    --plan-publisher                   : Plan publisher.

General Guidelines

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

Command Guidelines

[x] Each command and parameter has a meaningful description.
[x] Each new command has a test.

(see Authoring Command Modules)

@troydai, can you help review this PR? We have committed to external users that we will deliver through the upcoming release, so I prefer it can get in as early as possible.

Looking.


Fixed:#593 (Solved Too many re-renders)

About The PR

Fixed : #593
Solved the issue of too many re-renders during input on both  Login.jsx & Signup.jsx

Introduced Changes :

Implemented better state management using Recoil
Used useRef to reduce the re-render on input.

Screenshots :
Before

After

Summary by CodeRabbit

New Features

Introduced Recoil for state management in authentication components.

Refactor

Updated authentication buttons and login form to use Recoil states for better state management.
Split the Login component's form logic into a new LoginForm component.

UI Updates

Minor UI adjustments in the login and signup forms for a cleaner user interface.

Chores

Wrapped the main app component with <RecoilRoot> for Recoil state management support.

I see you have added the recoil package, we were already using the built in Context API for state management up until now that wraps the app.jsx conponent.
Your Recoil rool sits on top of that, have you checked for conflicts between them ?

also @anshul-kh you have added the recoil package in the root directory, package.json also

I see you have added the recoil package, we were already using the built in Context API for state management up until now that wraps the app.jsx conponent. Your Recoil rool sits on top of that, have you checked for conflicts between them ?

I have tested it on my end , I don't get any issues. Is there any specific error, you are talking about.

also @anshul-kh you have added the recoil package in the root directory, package.json also

Sorry about that, I'll fix it now..

also @anshul-kh you have added the recoil package in the root directory, package.json also

Sorry about that, I'll fix it now..

no worries, I'm just checking it on my end. Will let you know if I face an issues.

no worries, I'm just checking it on my end. Will let you know if I face an issues.

That's great ,  Thanks .


Unsure what object-literal-shorthand rule is reporting here

I just updated to TSLint 3.15.0 and the new rule object-literal-shorthand seems to complain about this code:
const fullFilePath = sourceFile.fileName;
const lastSlashIdx = fullFilePath.lastIndexOf("/");
const filePath = fullFilePath.substr(0, lastSlashIdx + 1);
const fileName = fullFilePath.substr(lastSlashIdx + 1, fullFilePath.length);
const fileNameParts = fileName.split(".");
return {
    fileName: fileName,
    fileNameParts: fileNameParts,
    filePath: filePath
};

For each line in the return statement it throws: Expected property shorthand in object literal.
I'm not really sure what "shorthand" means in this context... and there is no documentation for this rule right now.

It seems that moving things from the variable declaration directly into the object will fix some of these warnings, but I can only do so much.
const fullFilePath = sourceFile.fileName;
const lastSlashIdx = fullFilePath.lastIndexOf("/");
const fileName = fullFilePath.substr(lastSlashIdx + 1, fullFilePath.length);
return {
    fileName: fileName,
    fileNameParts: fileName.split("."),
    filePath: fullFilePath.substr(0, lastSlashIdx + 1)
};

Doing that now only makes it throw one error for the first line of the return statement, but if I do what I think it's asking me to do here, it make my code more complex, which is not something I want to do.

This code satisfies the linter, but lines 3 & 5 are now duplicated :sob:
const fullFilePath = sourceFile.fileName;
const lastSlashIdx = fullFilePath.lastIndexOf("/");
const fileName = fullFilePath.substr(lastSlashIdx + 1, fullFilePath.length);
return {
    fileName: fullFilePath.substr(lastSlashIdx + 1, fullFilePath.length),
    fileNameParts: fileName.split("."),
    filePath: fullFilePath.substr(0, lastSlashIdx + 1)
};

I'll have to turn the rule off for now.

it's similar to this eslint rule: http://eslint.org/docs/rules/object-shorthand
more docs: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer
we just updated the docs a few min ago: http://palantir.github.io/tslint/rules/object-literal-shorthand/
feel free to send PRs to improve that rule's metadata :)

So this is the intended behavior then?

yes. you would probably be best off satisfying the rule this way:
return {
    fileName,
    fileNameParts,
    filePath
};

ok, interesting.  I think I understand now.,  I'll have to look into this more. Thanks & sorry to both with a non-bug!


build: update build target from ES5 to ES6

@pionxzh next step is to use ESNext?

@rtritto wait, what's your definition of ESNext? This is V3 branch. I want to drop the support for IE, so I update the target to ES6.

The special ESNext value refers to the highest version your version of TypeScript supports. This setting should be used with caution, since it doesn’t mean the same thing between different TypeScript versions and can make upgrades less predictable.
Source: https://www.typescriptlang.org/tsconfig#target

I think ESNext is risky...
I just checked, as the fact that we are using MUI, we should be able to directly follow their browserslistrc range?

Nice. 80 KB -> 63 KB 😄


Punitive Counterstrike text shows old and new trace formatting on top of each other

https://netrunnerdb.com/en/card/31078
New trace style (expected): Trace[5]. If successful...
Old trace style (still seen on cards that haven't been reprinted or added to the Card Text Updates document): Trace [5] – If successful...
Punitive currently displays as: Trace [5] –. If successful...
Probably there's just an extraneous bit of markup in the card data. This error does not currently appear for any other card.
Edit: fixed en dashes

Ah, interesting.
This isn't a problem with the card data.  This looks to be a rare instance (only?) of a Trace effect just being a full sentence.
The controller currently replaces <trace>Trace 5</trace> like this:
 $cardinfo['text'] = preg_replace('/<trace>([^<]+) ([X\d]+)<\/trace>/', '<strong>\1 [\2]</strong>–', $cardinfo['text']);
To fix this, we would actually need to update all other traces in the JSON to add the en dash as a literal and then remove that en dash from the controller.

I think you're overcomplicating it a bit. All you need to do is remove the <trace> tag that applies the old formatting. All these cards display the new formatting just fine: https://netrunnerdb.com/find/?q=x%3A"trace["&sort=name&view=text&_locale=en

Those don't actually use the trace syntax.  They just have a literal Trace[#] in the text, like Executive Function.
Compare:
    "text": "[subroutine] Trace[4]. If successful, do 1 brain damage.",
to CtM's:
 "text": "The first time the Runner trashes an installed Corp card each turn, you may <trace>trace 4</trace> If successful, give the Runner 1 tag (cannot be avoided).",

incidentally, this also makes them normal weight and not bold.

if we have a trace syntax, why use it sometimes, but not other times?  Is trace a bit of text or an ability with a keyword/formatting?  Right now it's 🤷   It would be better for data quality to have it be a real keyword so we can match on the ability. The text formatting is pretty inconsistent now with respect to what is a keyword and what is some literal business shoved in (hello html tags grumble grumble).

I noticed today that this issue now affects all trace cards updated in CTU1.3. Those cards are:
Power Grid Overload
Data Hound
Midseason Replacements
Hellion Alpha Test
Information Overload
Archangel
Turnpike
Waiver
Hard-Hitting News
Hellion Beta Test
IP Block
Thoth
Forced Connection
Trojan Horse
Kill Switch
Data Raven
SEA Source
and Punitive Counterstrike is still affected as well.

Those don't actually use the trace syntax. They just have a literal Trace[#] in the text, like Executive Function. Compare:  "text": "[subroutine] Trace[4]. If successful, do 1 brain damage.",
to CtM's:  "text": "The first time the Runner trashes an installed Corp card each turn, you may <trace>trace 4</trace> If successful, give the Runner 1 tag (cannot be avoided).",

All the cards listed in my previous comment should match the Executive Functioning style. Normal weight is correct.
For reference, here are all cards that have correct new-style trace formatting:
Sherlock 1.0
Ichi 2.0
Viktor 2.0
Self-destruct
Vikram 1.0
Sherlock 2.0
SYNC BRE
Warroid Tracker
Threat Level Alpha
Executive Functioning
Holmegaard
Black Hat
Ichi 1.0
And here are all cards that still use old trace formatting, and this is expected because they haven't yet appeared in a CTU release:
Matrix Analyzer
Restructured Datapool
TMI
Draco
Viper
Uroboros
Salvage
Foxfire
Burke Bugs
Invasion of Privacy
Muckraker
Snoop
Broadcast Square
Shinobi
Taurus
Manhunt
Sagittarius
Gemini
Snatch and Grab
Troll
Virgo
Checkpoint
Searchlight
Gutenberg
News Hound
Resistor
Assassin
Security Nexus
Another Day, Another Paycheck
NBN: Controlling the Message
Enforcing Loyalty
Hatchet Job
Door to Door
Citadel Sanctuary
Drone Screen
Veritas
Macrophage
The Archivist
Traffic Analyzer
Bernice Mai
Shadow
No One Home
White Hat
Kuwinda K4H1U3
Tempus
Amani Senai
Intake
Surveyor
SIU
Eavesdrop
Ash 2X3ZB9CY
Flare
Caduceus
Paper Trail
Hunter
Trebuchet
Scapenet
Winchester

if we have a trace syntax, why use it sometimes, but not other times? Is trace a bit of text or an ability with a keyword/formatting? Right now it's 🤷 It would be better for data quality to have it be a real keyword so we can match on the ability. The text formatting is pretty inconsistent now with respect to what is a keyword and what is some literal business shoved in (hello html tags grumble grumble).

Yeah, it's inconsistent. The card pool as a whole is inconsistent in this and several other ways, because the Rules team does not have bandwidth to update card text for the entire backlog at once. We'll get there eventually, and then the old trace formatting will be fully obsolete. If you want to be done with the HTML tag before then, I have no objection to replacing e.g. <trace>trace 4</trace> with Trace[4] -   for all cards that have not been updated.
Actually maybe I'm misunderstanding what you ultimately want. What do you mean by a "real keyword"? Would eliminating the <trace> tag entirely be a good thing or a bad thing from your point of view?

I will look into this one. This is my personal preference:

changing all the cards not using <trace> tag to use them for consistency
update the <trace> card format to fit the new styling

In my opinion, keep the <trace> tag is much better in term of searching down the road since we can use it to easily identify the <trace> keyword with just a normal random trace word in the paragraph. What do you guys think?

also, @plural, I'm seeing that we are doing <trace>Trace 5</trace>. If this is meant to use for <trace> only, would you mind if I change the format to just <trace>5</trace> instead? I can make the code still work with any cards that have the old format in it.

I suppose there is a case to be made for being able to search cards that perform a trace separately from cards that mention trace. How does scryfall implement "keyword:" searches? I'll defer to y'all in the trenches as to whether it's better to have additional markup in the text field or have extra fields.
I do think it could create weirdness down the road to have the internal representation diverge from the actual text, as the previous comment proposes. Would having just the <trace> tag without the non-tag occurence of trace in the text mess up search strings? Suppose for example I want to find cards that can optionally perform a trace. My first thought for the search to use would be x:"may trace".

Pretext: I'm totally new to netrunner so there r so many cards that I'm not aware of.
My goal is to distinguish between card effects like this when you trace, you might do abc with Trace [5]. If successful, do something. So if you want to search for something with the trace keyword, then x: trace would work perfectly. I'm not sure whether we have something like this implemented, but I'm assuming we can do something like a:trace (a being action, for example) to search for cards that allow you to have the action of trace

Welcome!
Practically speaking, there are only a handful of cards with "when you trace" and similar effects, and NSG has moved away from actively using the trace mechanic to begin with, so there aren't likely to be more of those cards any time soon. I do find myself searching for all kinds of weird things for Rules team work, so I would probably get some use out of a specialized functionality like you are suggesting. But it's also pretty easy to search x:trace t:"If successful"|"If unsuccessful", so maybe it isn't worth the implementation effort? I recommend following @plural's lead here

at this point, for the scope of this issue, there won't be any implementation for the searching one. I just put that out there to prove my point of having a ` tag should benefit us much better than leaving them as normal text.
The code change for this style change is quite minimal and I have already done it. I'm working on some data changes to switch the plain text of Trace[x] to use <trace> tag instead.


Question: does h5netcdf bring in the entire data from a netCDF file on a remote disk (like S3)?

Hi guys, first off I wanted to thank you for building this library, I have recently (just today) came across it, and it does the job for my particular case: I am opening a netCDF4 file located on an S3 FS (in a bucket), and then using h5netcdf to have it like a netCDF file; netCDF4-python complained that the file is not "local", but h5netcdf is very nice and accomodating with the file sitting in an S3 bucket. See the minimal code below:
    fs = s3fs.S3FileSystem(key=S3_ACCESS_KEY,  # eg "minioadmin" for Minio
                           secret=S3_SECRET_KEY,  # eg "minioadmin" for Minio
                           client_kwargs={'endpoint_url': S3_URL})  # eg "http://localhost:9000" for Minio
    with fs.open(uri, 'rb') as s3file:
        ds = h5netcdf.File(s3file, 'r', invalid_netcdf=True)
        print(f"Dataset loaded from S3 via h5netcdf: {ds}")

My very quick question to you, please: will ds (in this case), which is what the client sees of the netCDF file at location uri in the S3 bucket, contain the entire (very large, on occasion) data or is it just just a much smaller reference file with metadata, and just a pointer to the physical data? Very many thanks in advance :beer:

Generally h5netcdf/h5py only load minimum required data, which in principle should be using something like a range-get for S3FileSytem.
That said, HDF5 was designed for local filesystems, so sometimes reading metadata can be more expensive than one would think. But it certainly should not be loading the entire thing...

Ih @bnlawrence doesn't have other q's, I can close this - great timely communication, cheers, guys!


Unable to obtain shap values using embedding layer, tf 2.1.0 & shap 0.35.0

Full code using synthetic data to exemplify error message
import pandas as pd
import numpy as np
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import MSE
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Dense, Embedding, Flatten, Concatenate
from tensorflow.keras.backend import clear_session
from sklearn.model_selection import train_test_split
import shap
def convert_list(df):
'''
:param df: Pandas dataframe
:return: list of lists, with each categorical feature (as pd.Series) in its own list,
and one list (of pd.DataFrame) of numeric features
'''
return [df[col] for col in categorical_features] + [df[numeric_features]]
def expand_to_2d(arr):
'''
Expand a numpy array to two dimensions if it is one-dimensional.
:param arr: Numpy array
:return: expanded Numpy array
'''
if len(arr.shape) >= 2:
return arr
return np.expand_dims(arr, axis=1)
Synthesize data
np.random.seed(123)
cat1 = np.random.randint(low=1, high=5, size=100) # assume factorized categorical variable
cat2 = np.random.randint(low=1, high=10, size=100) # assume factorized categorical variable
num1 = np.random.random(size=100) # assume continuous variable that's been min-max normalized
num2 = np.random.random(size=100) # assume continuous variable that's been min-max normalized
outcome = np.random.randint(low=1, high=100, size=100)
df = pd.DataFrame({'cat1': cat1, 'cat2': cat2, 'num1': num1, 'num2': num2, 'outcome': outcome})
Train, validation sets
X_train, X_val, y_train, y_val = train_test_split(df, df['outcome'], test_size=0.25, random_state=123)
Identify categorical vs. numeric features in dataset
categorical_features = ['cat1']
numeric_features = ['num1']
X_train_list = convert_list(X_train)
X_val_list = convert_list(X_val)
Build model (Keras Model API)
clear_session()
categorical_input_layers = [Input(shape=(1,), dtype='int32') for _ in categorical_features]
embedded_layers = [Embedding(input_dim=df[col].nunique() + 1, output_dim=1)(lyr) for (col, lyr) in zip(categorical_features, categorical_input_layers)]
flatten_layer = [Flatten()(lyr) for lyr in embedded_layers]
numeric_input_layer = Input(shape=(len(numeric_features),))
concat_layer = Concatenate(axis=1)(flatten_layer + [numeric_input_layer])
dense_layer = Dense(128, activation='relu')(concat_layer)
output_layer = Dense(1, activation='linear', name='output')(dense_layer)
model = Model(inputs=categorical_input_layers+[numeric_input_layer], outputs=output_layer)
Fit model
model.compile(loss=MSE, optimizer=Adam(amsgrad=True), metrics=[MSE])
model.fit(X_train_list, y_train, epochs=50, batch_size=256, verbose=0, validation_data=(X_val_list, y_val))
Subsample & reshape data
sample = [i[:5] for i in X_train_list] # get first 5 obs of X_train
reshaped_sample = [expand_to_2d(i.values) for i in sample]
Get shap values
explainer = shap.DeepExplainer(model, reshaped_sample)
shap_values = explainer.shap_values(reshaped_sample) # breaks here
Traceback
WARNING:tensorflow:The dtype of the watched tensor must be floating (e.g. tf.float32), got tf.int32
WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\shap\explainers\deep_init_.py", line 119, in shap_values
return self.explainer.shap_values(X, ranked_outputs, output_rank_order, check_additivity=check_additivity)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\shap\explainers\deep\deep_tf.py", line 304, in shap_values
sample_phis = self.run(self.phi_symbolic(feature_ind), self.model_inputs, joint_input)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\shap\explainers\deep\deep_tf.py", line 361, in run
return self.execute_with_overridden_gradients(anon)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\shap\explainers\deep\deep_tf.py", line 397, in execute_with_overridden_gradients
out = f()
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\shap\explainers\deep\deep_tf.py", line 357, in anon
final_out = out(inputs)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 568, in call
result = self._call(*args, **kwds)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 615, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 497, in _initialize
*args, **kwds))
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\function.py", line 2389, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\function.py", line 2703, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\function.py", line 2593, in _create_graph_function
capture_by_value=self._capture_by_value),
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 978, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\def_function.py", line 439, in wrapped_fn
return weak_wrapped_fn().wrapped(*args, **kwds)
File "C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 968, in wrapper
raise e.ag_error_metadata.to_exception(e)
tensorflow.python.framework.errors_impl.InternalError: in converted code:
C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\shap\explainers\deep\deep_tf.py:244 grad_graph  *
x_grad = tape.gradient(out, shap_rAnD)
C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\backprop.py:1029 gradient
unconnected_gradients=unconnected_gradients)
C:\Users\jking\Documents\PythonVirtualEnv\20200301\lib\site-packages\tensorflow_core\python\eager\imperative_grad.py:77 imperative_grad
compat.as_str(unconnected_gradients.value))
InternalError: Recorded operation 'shap_ResourceGather' returned too few gradients. Expected 2 but received 3

Hi @kingjosephm,
Same problem here. Did you manage to solve it?

Hi @lrdsouza,
I unfortunately did not, but would welcome any suggestions! In another variant I also tested whether the problem was the dual input layers in the Keras Model class by feeding it separate lists of numeric-only features (and no embedding layer). This, however, worked. So it seems something related to the presence of the embedding layer.

I had the same issue.  The only temp work-around that I could find was to

train the model
use get_weights[0] to grab the weights of the embedding layer
Save the embeddings to the original dataset
Run a new model using the embeddings as normal features

@Expendable0 As you re-train the new model, I am not sure if the new model will be the same as the original model. Any better walk-around?

Hi @kingjosephm
were you able to solve this?

@abiudrodas - unfortunately no, I ran out of time to attempt the solution proposed by @Expendable0, nor have I investigated alternative solutions. As far as I'm aware it still remains an issue

I just got the same error. Apparently its not resolved yet.

我也遇到这个问题了，两年了还没有解决办法吗

I'm having trouble with the same issue to this date.

Same issue with the embedding layer


how can i run just user api without activate web page ui

hi
i wrote code for manage users and i want have userapi without exist a web page like this https://yourdomain.com/admin/
for many reason include security reason i don't want exist that page how can i run userapi without that page or remove it completely ?
i used this command for active my user api link
joker nico domain.com http://<IP_ADDRESS>:24402

If your brook server and userapi run on same machine. You can let brook server and userapi communicate through <IP_ADDRESS>:24402. This will not expose port 24402 to the pubic.
If your brook server and userapi run on same subnet. You can let brook server and userapi communicate through such as <IP_ADDRESS>:24402. This will not expose port 24402 to the public.
If your brook server and userapi must communicate through public network. You can set firewall on userapi machine to only allow brook server ip access port 24402.
If you only care about 'web page', just remove it https://github.com/TxThinkingInc/brook-user-system/blob/master/main.js#L45-L67 and https://github.com/TxThinkingInc/brook-user-system/blob/master/main.js#L82-L326

v20241012 released


Removing edpm_hosts_entries role

The role isn't used anymore, so we shouldn't keep it around.

recheck

recheck

recheck

recheck

/lgtm

/hold lets hold for now we have an issue with expose dns service on a 3 node cluster without dedicated workers. a workaround would be to use /etc/hosts.

/hold lets hold for now we have an issue with expose dns service on a 3 node cluster without dedicated workers. a workaround would be to use /etc/hosts.

@stuggi Is this still considered on hold?

/hold lets hold for now we have an issue with expose dns service on a 3 node cluster without dedicated workers. a workaround would be to use /etc/hosts.

@stuggi Is this still considered on hold?

@slagle I believe it turned out that DNS worked with a 3 node master/worker cluster after all.

/hold lets hold for now we have an issue with expose dns service on a 3 node cluster without dedicated workers. a workaround would be to use /etc/hosts.

@stuggi Is this still considered on hold?

@slagle I believe it turned out that DNS worked with a 3 node master/worker cluster after all.

its no longer required to hold it. there was a missconfig which resulted in the dns service to not be reachable.

/override rdoproject.org/github-check


Auto-detect type in sample annotations (and maybe variant annotations)

In R, when you load a data table, it auto-detects whether each column is a character vs. numeric type.  It would be super if this could be implemented in Hail.  I'm guessing it would take the form of "if none of the fields in the column contain special characters or letters, then it's numeric, else it's character," (but maybe it's not so straight forward, not so sure...).
Anyways, when you have over 30 annotations that are numeric, it's a bit of a pain to have to go through writing all the -t flag options in Hail, so if it could be auto-detected, that would be super!
In the case where 'dummy' variables are used (like 1-5 for Batch), then the user should be able to say that that's a string or a "factor" as it is in R (or a character/string, which is essentially the same), for the purposes of analysis in linear regression.

Tim added impute.  Impute has some problems, continue discussion here: https://github.com/broadinstitute/hail/issues/520.


Fix target_sample_hz typing for other uses

Update the other target_sample_hz types following the pattern in release 0.4.3 (since the default code in the README currently fails as-is, I think because target_sample_hz was previously set to a single int).

@LWprogramming oh, actually for each of those modules, they only use one target resampled frequency, so it is ok
the reason the sound dataset needs to be able to return multiple target resample frequencies is because the CoarseTransformer contains both token ids from the masked language model as well as soundstream, and those two systems have different resample frequencies

@LWprogramming which code in the readme is failing?

@LWprogramming oops, i forgot to allow it to also accept single integer values
want to try 0.4.4?

Awesome, that part works now! I was following the part about SoundStreamTrainer and can get it to run now :)

nice! hope you train some cool models :^)


Can't access ContextualData values in a Log4j2 "lookup"

I have successfully used ContextualData.put to get per-request information into normal log messages emitted with "vanilla" Log4J2 in the way described in the documentation. However, we use DataDog which prefers you to log JSON blobs. DD understands the default JSON layout provided by Log4j2 ( https://logging.apache.org/log4j/2.x/manual/layouts.html#JSONLayout ). However, to put an arbitrary value into the JSON blob emitted when you log, you have to use this format:
  <JsonLayout>
    <KeyValuePair key="additionalField1" value="constant value"/>
    <KeyValuePair key="additionalField2" value="$${ctx:key}"/>
  </JsonLayout>

The documentation linked above says that value only supports "lookups". If I understand the Log4j2 documentation, lookups are a separate concept from PatternConverters, which seems to be what this project uses.
Is it possible for me to get at the vcl object in a lookup?
(Also to be clear i did try all the obvious things like $${vcl:my_value:-} and they don't seem to work. Not that I would expect them to.)

The way I got this to work was to make my own plugin! Feel free to use it / modify it if you want
import io.reactiverse.contextual.logging.ContextualData;
import io.vertx.core.Vertx;
import io.vertx.core.impl.ContextInternal;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;

@Plugin(name = "vctx", category = StrLookup.CATEGORY)
public class VertxLog4j2Lookup implements StrLookup
{
    private static final String DEFAULT_VALUE = "(NONE)";

    /**
     * Lookup the value for the key.
     *
     * @param key the key to be looked up, may be null
     * @return The value for the key.
     */
    @Override
    public String lookup(String key)
    {
        final ContextInternal context = (ContextInternal) Vertx.currentContext();
        if (context != null && key != null)
        {
            return ContextualData.getOrDefault(key, DEFAULT_VALUE);
        }
        else
        {
            return DEFAULT_VALUE;
        }
    }

    /**
     * Lookup the value for the key using the data in the LogEvent.
     *
     * @param event The current LogEvent.
     * @param key   the key to be looked up, may be null
     * @return The value associated with the key.
     */
    @Override
    public String lookup(LogEvent event, String key)
    {
        final ContextInternal context = (ContextInternal) Vertx.currentContext();
        if (context != null && key != null)
        {
            return ContextualData.getOrDefault(key, DEFAULT_VALUE);
        }
        else
        {
            return DEFAULT_VALUE;
        }
    }
}

For anyone who ends up here via google, you have to modify your log4j2 configuration file to search the package containing this plugin. For example, an xml-style config might start like this:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" packages="ai.brace.util">
    <Appenders>
...

where the value for the "packages" key is the name of the package containing classes annotated with @Plugin

Thanks for reporting this @chris-brace.
It seems string lookup plugins work both with json and pattern layouts (possibly others).

Yeah! I honestly have no idea what is "normal" here. I have always been a little stumped as to what the community best practices are when releasing a library which needs to interoperate with all of the different java logging libs. I've always used log4j2 but even that is hard to play nice with when you're not the one controlling the configuration.
Right now im just trying to figure out if i can / if its a good idea to inject these data into the MDC object associated with each logevent :/

@chris-brace would you mind giving a try to #5 ?

Sure :) is there a version on maven or do i have to pull the project and set up the dependency?

You have to pull the project and build the snapshot version.

ok i will try to get to this today or tomorrow. thank you for your help!

Thank you!

I went ahead and merged the PR. I'm still interested in your feedback if you get a chance to try the snapshot version.

Sorry for the slow response. I have scheduled time to do this either friday or monday!

@chris-brace no worries, thank you!

Hi, FWIW I was able to test this and it works for me! Thank you! Do you know when you're planning on pushing the next release?
My configuration:
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = JsonLayout
appender.console.layout.compact = false
appender.console.layout.requestId.type = KeyValuePair
appender.console.layout.requestId.key = requestId
appender.console.layout.requestId.value = $${ctx:requestId:-}
appender.console.layout.caller.type = KeyValuePair
appender.console.layout.caller.key = caller
appender.console.layout.caller.value = $${ctx:caller:-}

Output:
{
  "instant" : {
    "epochSecond" :<PHONE_NUMBER>,
    "nanoOfSecond" : 164873000
  },
  "thread" : "vert.x-eventloop-thread-0",
  "level" : "INFO",
  "loggerName" : "gov.cms.mpsm.rda.MainVerticle",
  "message" : "Proccesing 1 Record",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
  "threadId" : 16,
  "threadPriority" : 5,
  "requestId" : "0d0e9df3-8022-40e8-bf90-efe50554cb4e",
  "caller" : "/0:0:0:0:0:0:0:1:48994"
}

@ferristseng great, thank you for the feedback! I hope I can release before the end of this month.

@ferristseng @chris-brace version 1.1.0 has been released and includes this fix.

Thank you!
On Mon, Sep 27, 2021 at 05:33 Thomas Segismont @.***>
wrote:

@ferristseng https://github.com/ferristseng @chris-brace
https://github.com/chris-brace version 1.1.0 has been released and
includes this fix.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reactiverse/reactiverse-contextual-logging/issues/4#issuecomment-927694440,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALGMKB7GZFAZCL3YVMHM2FDUEA3ALANCNFSM45BMLQ5A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--
Chris Galardi
Head of Enterprise Architecture
Brace | New York, NY
--
Confidentiality Notice: This email is intended for the sole use of the
intended recipient(s) and may contain confidential, proprietary or
privileged information. If you are not the intended recipient, you are
notified that any use, review, dissemination, copying or action taken based
on this message or its attachments, if any, is prohibited. If you are not
the intended recipient, please contact the sender by reply email and
destroy or delete all copies of the original message and any attachments.

Hey! Sorry to take so long to reply but when i include 1.1.0 as an implementation dependency i now get this when running the shadowjar
> Task :backend:servicer-api:runShadow
DEBUG StatusLogger Using ShutdownCallbackRegistry class org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry
DEBUG StatusLogger Not in a ServletContext environment, thus not loading WebLookup plugin.
DEBUG StatusLogger AsyncLogger.ThreadNameStrategy=UNCACHED (user specified null, default is UNCACHED)
TRACE StatusLogger Using default SystemClock for timestamps.
DEBUG StatusLogger org.apache.logging.log4j.core.util.SystemClock supports precise timestamps.
DEBUG StatusLogger Not in a ServletContext environment, thus not loading WebLookup plugin.
DEBUG StatusLogger Took 0.004526 seconds to load 1 plugins from jdk.internal.loader.ClassLoaders$AppClassLoader@443b7951
DEBUG StatusLogger PluginManager 'Converter' found 1 plugins
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
DEBUG StatusLogger Starting OutputStreamManager SYSTEM_OUT.false.false-1
DEBUG StatusLogger Initializing Thread Context Data Service Providers
DEBUG StatusLogger Thread Context Data Service Provider initialization complete
DEBUG StatusLogger Starting LoggerContext[name=443b7951<EMAIL_ADDRESS>DEBUG StatusLogger Reconfiguration started for context[name=443b7951] at URI null (org.apache.logging.log4j.core.LoggerContext@14acaea5) with optional ClassLoader: null
DEBUG StatusLogger Not in a ServletContext environment, thus not loading WebLookup plugin.
DEBUG StatusLogger PluginManager 'ConfigurationFactory' found 0 plugins
DEBUG StatusLogger Using configurationFactory org.apache.logging.log4j.core.config.ConfigurationFactory$Factory@42d80b78
TRACE StatusLogger Trying to find [log4j2.xml] using context class loader jdk.internal.loader.ClassLoaders$AppClassLoader@443b7951.
ERROR StatusLogger Reconfiguration failed: No configuration found for '443b7951' at 'null' in 'null'
DEBUG StatusLogger Shutdown hook enabled. Registering a new one.
DEBUG StatusLogger LoggerContext[name=443b7951, org.apache.logging.log4j.core.LoggerContext@14acaea5] started OK.

and no logging, of course. when i inspect the /META-INF/services directory inside the fatjar it contains the org.apache...ContextDataProvider file but the contents are org.apache.logging.log4j.core.impl.ThreadContextDataProvider. I believe something is conflicting and will investigate further.
I believe this is similar to https://issues.apache.org/jira/browse/LOG4J2-673 but not exactly the same.

I will not open a new issue yet because I believe this is due to a ShadowJar limitation i asked about here https://github.com/johnrengelman/shadow/issues/714

I will not open a new issue yet because I believe this is due to a ShadowJar limitation i asked about here johnrengelman/shadow#714

Hey, we actually ran into the same issue and found that it's an issue with fat/uber jars and multiple Plugins.dat files. For the maven shade plugin, we were able to use this transformer: https://github.com/edwgiz/maven-shaded-log4j-transformer

@ferristseng do you have any insight into how i can use this with shadowjar without re-writing the transformer in kotlin / groovy?

Oh i figured it out!
https://github.com/Ultranium/shadowjar-log4j-transform-reproducer/blob/master/app/build.gradle

...

tasks.withType<ShadowJar> {
    manifest {
        attributes(mapOf("Multi-Release" to "true"))
    }
    mergeServiceFiles {
        include("META-INF/services/io.vertx.core.spi.VerticleFactory")
        include("META-INF/services/org.apache.logging.log4j.core.util.ContextDataProvider")
        transform(Log4j2PluginsCacheFileTransformer())
    }
}


chat: images pasted from clipboard can be "sent" before they've actually been uploaded.

This is similar to the issue we were seeing with the dialog uploads.

Dupe of #1642


Handle rebalances in the producer

After a rebalance the producer fails to send the messages
with 'Unknown topic or partition' error.
By forcing a cluster metadata refresh after rescuing this error the
producer adapts to the new cluster state and continues to send
the messages normally.
I reproduce the problem with the following steps:
I run the following producer script:
require "kafka"

kafka = Kafka.new(seed_brokers: ["kafka.vm.skroutz.gr:9092", "kafka-b.vm.skroutz.gr:9092"],
                  logger: Logger.new(STDOUT))

producer = kafka.async_producer(delivery_interval: 10)

while true
  producer.produce("hello", topic: "skroutz.test")
  sleep 5
end

Cluster state before rebalance:

Logger output before rebalance:
D, [2016-05-26T15:28:58.379706 #11682] DEBUG -- : Current leader for skroutz.test/1 is node kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:28:58.379785 #11682] DEBUG -- : Current leader for skroutz.test/2 is node kafka.vm.skroutz.gr:9092 (node_id=1)
I, [2016-05-26T15:28:58.379806 #11682]  INFO -- : Sending 1 messages to kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:28:58.379833 #11682] DEBUG -- : Sending request 2 to kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:28:58.379920 #11682] DEBUG -- : Waiting for response 2 from kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:28:58.382325 #11682] DEBUG -- : Received response 2 from kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:28:58.382374 #11682] DEBUG -- : Successfully appended 1 messages to skroutz.test/1
I, [2016-05-26T15:28:58.382394 #11682]  INFO -- : Sending 1 messages to kafka.vm.skroutz.gr:9092 (node_id=1)
D, [2016-05-26T15:28:58.382413 #11682] DEBUG -- : Sending request 5 to kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:28:58.382507 #11682] DEBUG -- : Waiting for response 5 from kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:28:58.384264 #11682] DEBUG -- : Received response 5 from kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:28:58.384317 #11682] DEBUG -- : Successfully appended 1 messages to skroutz.test/2

Now I trigger a rebalance.
Cluster state after rebalance:

Logger output after rebalance:
D, [2016-05-26T15:30:08.380421 #11682] DEBUG -- : Current leader for skroutz.test/1 is node kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:30:08.380483 #11682] DEBUG -- : Current leader for skroutz.test/0 is node kafka-b.vm.skroutz.gr:9092 (node_id=2)
D, [2016-05-26T15:30:08.380509 #11682] DEBUG -- : Current leader for skroutz.test/2 is node kafka.vm.skroutz.gr:9092 (node_id=1)
I, [2016-05-26T15:30:08.380527 #11682]  INFO -- : Sending 4 messages to kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:30:08.380557 #11682] DEBUG -- : Sending request 10 to kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:30:08.380671 #11682] DEBUG -- : Waiting for response 10 from kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:30:08.382102 #11682] DEBUG -- : Received response 10 from kafka-c.vm.skroutz.gr:9092
E, [2016-05-26T15:30:08.382194 #11682] ERROR -- : Unknown topic or partition skroutz.test/1
I, [2016-05-26T15:30:08.382214 #11682]  INFO -- : Sending 1 messages to kafka-b.vm.skroutz.gr:9092 (node_id=2)
D, [2016-05-26T15:30:08.382233 #11682] DEBUG -- : Sending request 9 to kafka-b.vm.skroutz.gr:9092
D, [2016-05-26T15:30:08.382283 #11682] DEBUG -- : Waiting for response 9 from kafka-b.vm.skroutz.gr:9092
D, [2016-05-26T15:30:08.383721 #11682] DEBUG -- : Received response 9 from kafka-b.vm.skroutz.gr:9092
E, [2016-05-26T15:30:08.383769 #11682] ERROR -- : Unknown topic or partition skroutz.test/0
I, [2016-05-26T15:30:08.383786 #11682]  INFO -- : Sending 1 messages to kafka.vm.skroutz.gr:9092 (node_id=1)
D, [2016-05-26T15:30:08.383805 #11682] DEBUG -- : Sending request 11 to kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:30:08.383889 #11682] DEBUG -- : Waiting for response 11 from kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:30:08.384873 #11682] DEBUG -- : Received response 11 from kafka.vm.skroutz.gr:9092
E, [2016-05-26T15:30:08.384921 #11682] ERROR -- : Unknown topic or partition skroutz.test/2
W, [2016-05-26T15:30:08.384937 #11682]  WARN -- : Failed to send all messages; attempting retry 1 of 2 after 1s
D, [2016-05-26T15:30:09.385085 #11682] DEBUG -- : Current leader for skroutz.test/1 is node kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:30:09.385144 #11682] DEBUG -- : Current leader for skroutz.test/0 is node kafka-b.vm.skroutz.gr:9092 (node_id=2)
D, [2016-05-26T15:30:09.385199 #11682] DEBUG -- : Current leader for skroutz.test/2 is node kafka.vm.skroutz.gr:9092 (node_id=1)
I, [2016-05-26T15:30:09.385218 #11682]  INFO -- : Sending 4 messages to kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:30:09.385245 #11682] DEBUG -- : Sending request 11 to kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:30:09.385352 #11682] DEBUG -- : Waiting for response 11 from kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:30:09.386760 #11682] DEBUG -- : Received response 11 from kafka-c.vm.skroutz.gr:9092
E, [2016-05-26T15:30:09.386820 #11682] ERROR -- : Unknown topic or partition skroutz.test/1
I, [2016-05-26T15:30:09.386836 #11682]  INFO -- : Sending 1 messages to kafka-b.vm.skroutz.gr:9092 (node_id=2)
D, [2016-05-26T15:30:09.386854 #11682] DEBUG -- : Sending request 10 to kafka-b.vm.skroutz.gr:9092
D, [2016-05-26T15:30:09.386919 #11682] DEBUG -- : Waiting for response 10 from kafka-b.vm.skroutz.gr:9092
D, [2016-05-26T15:30:09.388319 #11682] DEBUG -- : Received response 10 from kafka-b.vm.skroutz.gr:9092
E, [2016-05-26T15:30:09.388368 #11682] ERROR -- : Unknown topic or partition skroutz.test/0
I, [2016-05-26T15:30:09.388385 #11682]  INFO -- : Sending 1 messages to kafka.vm.skroutz.gr:9092 (node_id=1)
D, [2016-05-26T15:30:09.388404 #11682] DEBUG -- : Sending request 12 to kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:30:09.388465 #11682] DEBUG -- : Waiting for response 12 from kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:30:09.389644 #11682] DEBUG -- : Received response 12 from kafka.vm.skroutz.gr:9092
E, [2016-05-26T15:30:09.389692 #11682] ERROR -- : Unknown topic or partition skroutz.test/2
W, [2016-05-26T15:30:09.389709 #11682]  WARN -- : Failed to send all messages; attempting retry 2 of 2 after 1s
D, [2016-05-26T15:30:10.389868 #11682] DEBUG -- : Current leader for skroutz.test/1 is node kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:30:10.389939 #11682] DEBUG -- : Current leader for skroutz.test/0 is node kafka-b.vm.skroutz.gr:9092 (node_id=2)
D, [2016-05-26T15:30:10.389975 #11682] DEBUG -- : Current leader for skroutz.test/2 is node kafka.vm.skroutz.gr:9092 (node_id=1)
I, [2016-05-26T15:30:10.389993 #11682]  INFO -- : Sending 4 messages to kafka-c.vm.skroutz.gr:9092 (node_id=3)
D, [2016-05-26T15:30:10.390023 #11682] DEBUG -- : Sending request 12 to kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:30:10.390130 #11682] DEBUG -- : Waiting for response 12 from kafka-c.vm.skroutz.gr:9092
D, [2016-05-26T15:30:10.391432 #11682] DEBUG -- : Received response 12 from kafka-c.vm.skroutz.gr:9092
E, [2016-05-26T15:30:10.391499 #11682] ERROR -- : Unknown topic or partition skroutz.test/1
I, [2016-05-26T15:30:10.391515 #11682]  INFO -- : Sending 1 messages to kafka-b.vm.skroutz.gr:9092 (node_id=2)
D, [2016-05-26T15:30:10.391534 #11682] DEBUG -- : Sending request 11 to kafka-b.vm.skroutz.gr:9092
D, [2016-05-26T15:30:10.391589 #11682] DEBUG -- : Waiting for response 11 from kafka-b.vm.skroutz.gr:9092
D, [2016-05-26T15:30:10.393046 #11682] DEBUG -- : Received response 11 from kafka-b.vm.skroutz.gr:9092
E, [2016-05-26T15:30:10.393096 #11682] ERROR -- : Unknown topic or partition skroutz.test/0
I, [2016-05-26T15:30:10.393112 #11682]  INFO -- : Sending 1 messages to kafka.vm.skroutz.gr:9092 (node_id=1)
D, [2016-05-26T15:30:10.393131 #11682] DEBUG -- : Sending request 13 to kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:30:10.393184 #11682] DEBUG -- : Waiting for response 13 from kafka.vm.skroutz.gr:9092
D, [2016-05-26T15:30:10.394341 #11682] DEBUG -- : Received response 13 from kafka.vm.skroutz.gr:9092
E, [2016-05-26T15:30:10.394390 #11682] ERROR -- : Unknown topic or partition skroutz.test/2
E, [2016-05-26T15:30:10.394407 #11682] ERROR -- : Failed to send all messages; keeping remaining messages in buffer

:heart:


[REQUEST] 集成XTuner

🤩 Features description [Please make everyone to understand it]
与XTuner框架集成

直接使用mmengine的集成方式，运行xtuner train时，会报错：
KeyError: 'SwanlabVisBackend is not in the mmengine::vis_backend registry. Please check whether the value of `SwanlabVisBackend` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

https://docs.swanlab.cn/zh/guide_cloud/integration/integration-xtuner.html


Tests

Kevin, I don't think you have uploaded the branch you are working on:
(dpanonymize1) [tb571@rc-predict-lochness tests]$ tree
.
├── dpanonymize
│   ├── test_dpanonymize.py
│   └── test_redcap.py
├── dpanonymize_test.py
└── scripts
    └── test_dpanon.py

(dpanonymize1) [tb571@rc-predict-lochness tests]$ pytest -v dpanonymize_test.py
=================================== test session starts ====================================
platform linux -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /home/tb571/miniconda3/envs/dpanonymize1/bin/python
cachedir: .pytest_cache
rootdir: /home/tb571/dpanonymize
collected 2 items

dpanonymize_test.py::test_test_phoenix_structure_short PASSED                        [ 50%]
dpanonymize_test.py::test_test_phoenix_structure_BIDS PASSED                         [100%]

==================================== 2 passed in 0.60s =====================================

bash dpanonymize_test.sh

This instruction is invalid for the current master.

Please run
cd dpanonymize/tests
bash dpanonymize_test.sh

Okay let me try again--can you point out the dpanonymize_test.sh script in the master branch?

My bad, just added.

Now I see what you were doing. There are still issues, I shall fix them for us.

Is this supposed to install pytest when tests are run?

TODO: test a few commands from https://github.com/AMP-SCZ/dpanonymize#how-to-run

Is this supposed to install pytest when tests are run?

no
https://stackoverflow.com/questions/15422527/best-practices-how-do-you-list-required-dependencies-in-your-setup-py

This is interesting. So before running tests, one should:

python setup.py test

?

I have not built the test to work from the setup.py for dpanonymize. We should use
cd dpanonymize/tests
bash dpanonymize_test.sh

Sorry, I have not been clear--since pytest package is missing, your we should use won't run the tests.

pip install pytest to be able to run the tests.
sounds good


Updated link to project license in Readme.md

The link to the project license was incorrect and returning a 404 error so I updated it to point to the correct file.

Coverage remained the same at 100.0% when pulling 0bf440b107b0fe8a26e9f3117ada2b9f66cb829b on kickes:master into 58ada80d9849f0f86f942b423dda630d1945b535 on ruipgil:master.

Good catch, thanks!