Recommend using row_security = off for starting up with RLS

Problem
RLS filters rows silently, this trips a lots of newcomers when trying PostgREST on a table with RLS enabled. See https://github.com/PostgREST/postgrest/discussions/3189
Solution
Recommend using row_security, this will give a clear error when enabled.
alter role postgrest_test_anonymous set row_security to 'off';
alter table projects enable row level security;

curl localhost:3000/projects

HTTP/1.1 401 Unauthorized
{"code":"42501","details":null,"hint":null,"message":"query would be affected by row-level security policy for table \"projects\""}

Perhaps we should add a Prefer: row-security so users can activate it per request?

Yes, it's only a debugging hint. I'm not sure what's the right way, I also thought of a special response header like:
GET /projects

X-Pgrst-Affected-By-RLS: true
[]


The node-gyp compile error message is not helpful, could anyone take a look?

error c2240 cannot convert from 'v8::Value *' to 'v8:object *volatile'
the error points to ".node-gyp\4.4.5\include\node\v8.h(221)", so I have no idea what's goes wrong.
p.s. compiled on a windows machine.

That error suggests a bug in the module you're trying to compile.  You don't mention what module it is but I'd file a bug against that project.  Good luck.


Feature/add return types for before output rendered response mutator hook

Closes #5202.
We have to add return types for beforeOutputRenderedResponse. Otherwise, developers might forget to return value (e.g. they want only to operate on headers) - with types they will be instantly informed about that!
Which Environment This Relates To

[x] Test version (https://test.storefrontcloud.io) - this is a new feature or improvement for Vue Storefront. I've created branch from develop branch and want to merge it back to develop

Upgrade Notes and Changelog

[x] No upgrade steps required (100% backward compatibility and no breaking changes)
[ ] I've updated the Upgrade notes and Changelog on how to port existing Vue Storefront sites with this new feature

IMPORTANT NOTICE - Remember to update CHANGELOG.md with description of your change
Contribution and Currently Important Rules Acceptance

[x] I read and followed contribution rules

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.5 out of 8 committers have signed the CLA.:white_check_mark: gibkigonzo:white_check_mark: Fifciu:white_check_mark: didkan:white_check_mark: TamTran72111:white_check_mark: dixitdeepak:x: jahvi:x: talalus:x: lsliwaradioluzYou have signed the CLA already but the status is still pending? Let us recheck it.

Hello @lsliwaradioluz. You created PR for bad branch. Please make it for hotfix/v1.12.3 or develop


E/H264Packet: waiting for a valid sps and pps

Hi @pedroSG94, First I'm greatful of this library it helped me wind up my streaming problems.
I have an android app that integrates a USB camera (Otoscope) using a UVC usb library but without RTMP support. The usb camera library (written in java) supports video encoder and exports chunk of videos which is ByteArray, and with offset, length, timestamp.
I want to stream this chunk of video data into my RTMP server (nginx). Using the RTMPClient class, I called sendVideo I sent the video data as:
val info = MediaCodec.BufferInfo()
info.offset = offset
info.size = length
info.presentationTimeUs = timestamp
rtmpClient.sendVideo(ByteBuffer.wrap(data), info)

But I'm getting
I/RtmpSender: Skipping iteration, frame null
E/H264Packet: waiting for a valid sps and pps
E/H264Packet: waiting for a valid sps and pps
E/H264Packet: waiting for a valid sps and pps
E/H264Packet: waiting for a valid sps and pps
E/H264Packet: waiting for a valid sps and pps
E/H264Packet: waiting for a valid sps and pps
E/H264Packet: waiting for a valid sps and pps

Hello,
You missed flag value in MediaCodec.BufferInfo class. Also, you need set sps and pps to RtmpClient class to init stream. You have an issue here that is like your case:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/issues/1033
RtspClient and RtmpClient usage is the same so you can apply all for this case.


unable to start deis-builder

We're using a custom cloud-config to spin up hosts. Closing this now.


Could not create decoration from factory! Running with no decorations.

Running any example program in Wayland,

Could not create decoration from factory! Running with no decorations.

And window title bar does not appear.
https://code.qt.io/cgit/qt/qtwayland.git/tree/src/client/qwaylandwindow.cpp?h=5.15#n853

Setting QT_QPA_PLATFORM is a workaround.
QT_QPA_PLATFORM=xcb python examples/tutorial.py


webpack-dev-server breaks on changing folder structure or renaming files and requires restart

Do you want to request a feature or report a bug?
In my opinion this is a bug because it was working in older versions of webpack.

What is the current behavior?
When I rename some file or change folder structure the webpack starts showing "...Module build failed: Error: ENOENT: no such file or directory,...", and it doesn't recover from that, so I have to restart webpack dev server in order to have the hot reloading working again. The documentation suggests explicitly setting watchOptions but didn't work. Is there some cache invalidation that I have to configure or...?
If the current behavior is a bug, please provide the steps to reproduce.
Described above and my webpack dev configuration is:
const path = require('path');
const webpack = require('webpack');
const config = require('../config');

module.exports = {
  entry: [
    'react-hot-loader/patch',
    'webpack-dev-server/client?http://localhost:9001',
    'webpack/hot/only-dev-server',
    path.join(config.projectPath, 'src', 'reactApp'),
  ],
  output: {
    path: path.join(config.projectPath, 'assets'),
    filename: 'bundle.js',
    publicPath: 'http://localhost:9001/',
  },
  devtool: 'cheap-module-eval-source-map',
  plugins: [
    new webpack.HotModuleReplacementPlugin(),
    new webpack.NamedModulesPlugin(),
    new webpack.DefinePlugin({
      __DEV__: true,
      'process.env.NODE_ENV': JSON.stringify('development'),
    }),
    new webpack.ProvidePlugin({ Promise: 'bluebird' }),
  ],
  resolve: {
    extensions: ['.js', '.jsx'],
  },
  module: {
    noParse: /node_modules\/aws-sdk\/dist\/aws-sdk/,
    rules: [...],
  },
};

And my server.js file:
const devServer = new WebpackDevServer(webpack(webpackConfig), {
  hot: true,
  watchOptions: { poll: true },
  host: 'localhost',
  port: 9001,
  publicPath: 'http://localhost:9001/',
  compress: true,
  noInfo: true,
  stats: { colors: true, errorDetails: true },
  headers: { 'Access-Control-Allow-Origin': '*' },
  historyApiFallback: true,
});

devServer.listen(9001);

What is the expected behavior?
Like in older versions of webpack, the dev-server should not crash, but pick up the new folder structure.
If this is a feature request, what is motivation or use case for changing the behavior?
having to manually restart dev server on any file structure change and wait for it to build a bundle is not useful at all.
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
I'm using webpack 3.3.0 and webpack-dev-server 2.6.1, latest Chrome browser and Ubuntu 14.04.

The resolvers cache is a bit unsafe to modifications of already successfully resolved modules. You could disable it with resolve.unsafeCache: false but that would make it slow. You probably have to live with it.

@sokra
will try. thanks!
this was working well in webpack 1, right?
how come it changed?

Maybe because of more caching in newer version...

It really would be nice if it was possible to restart the build from scratch without exiting the watcher (think Ctrl-F5 for the watcher). Bonus points if it can re-parse the config it was started with as well. 🥇

@ingelity
Have you found a solution for your problem? Because I have same issue. #5774
Unfortunately I haven't found solution.

This worked for me:
watchOptions: {
  aggregateTimeout: 300,
  poll: 1000
}

thanks to SO

@ErichBSchulz Which webpack version do you use? Could you post your config file?

Wow I've been pulling my hair out trying to figure out what was causing my issue. This was the answer. Thank you @ingelity.
For the maintainers:
    "webpack": "^4.17.1",
    "webpack-dev-middleware": "^3.2.0",
    "webpack-dev-server": "^3.1.5",

Has the same problem when use webpack@2. But after upgrading webpack to version 3, fixed.

Problem still exists? If yep, please create minimum reproducible test repo, thanks!

@evilebottnawi for me the easiest way to reproduce was trying to change file extension, so another loader will be applied, but cache will not be invalidated for the previous loader".
Repo with repro is here https://github.com/Den-dp/webpack-issue-5523-repro and steps are in README too.

@Den-dp thanks

@webpack-bot move to webpack/webpack-dev-server

@Den-dp thanks for reproducible test repo, can you open new issue with example? Looks you find bug, problem not related to webpack-dev-server

@evilebottnawi you were quicker! Thanks ;)

I had this problem, I tried to clear the cache but nothing, I solved it by killing the node process, running again it was solved
killall node


can't find -lblas or -lcblas

current git version isn't working on fedora F24.  I have openblas, and could possibly link with mkl blas, but no cblas
pythran3 mag_sqr.cpp
WARNING: Disabling color, you really want to install colorlog.
/usr/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
/tmp/tmppprhr2hh/tmp/tmppprhr2hh/source.o: In function `main':
source.c:(.text+0x96): undefined reference to `cblas_ddot'
collect2: error: ld returned 1 exit status
/tmp/tmppprhr2hh/tmp/tmppprhr2hh/source.o: In function `main':
source.c:(.text+0x96): undefined reference to `cblas_ddot'
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "/usr/lib64/python3.5/distutils/unixccompiler.py", line 204, in link
    self.spawn(linker + ld_args)
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/ccompiler.py", line 27, in <lambda>
    m = lambda self, *args, **kw: func(self, *args, **kw)
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/ccompiler.py", line 71, in CCompiler_spawn
    raise DistutilsExecError('Command "%s" failed with exit status %d%s' % (cmd, s, msg))
distutils.errors.DistutilsExecError: Command "cc /tmp/tmppprhr2hh/tmp/tmppprhr2hh/source.o -L/usr/lib64 -lcblas -o /tmp/tmppprhr2hh/a.out" failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/system_info.py", line 1716, in has_cblas
    extra_postargs=info.get('extra_link_args', []))
  File "/usr/lib64/python3.5/distutils/ccompiler.py", line 734, in link_executable
    debug, extra_preargs, extra_postargs, None, target_lang)
  File "/usr/lib64/python3.5/distutils/unixccompiler.py", line 206, in link
    raise LinkError(msg)
distutils.errors.LinkError: Command "cc /tmp/tmppprhr2hh/tmp/tmppprhr2hh/source.o -L/usr/lib64 -lcblas -o /tmp/tmppprhr2hh/a.out" failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.5/distutils/unixccompiler.py", line 204, in link
    self.spawn(linker + ld_args)
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/ccompiler.py", line 27, in <lambda>
    m = lambda self, *args, **kw: func(self, *args, **kw)
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/ccompiler.py", line 71, in CCompiler_spawn
    raise DistutilsExecError('Command "%s" failed with exit status %d%s' % (cmd, s, msg))
distutils.errors.DistutilsExecError: Command "cc /tmp/tmppprhr2hh/tmp/tmppprhr2hh/source.o -L/usr/lib64 -lblas -o /tmp/tmppprhr2hh/a.out" failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nbecker/.local/bin/pythran3", line 9, in <module>
    load_entry_point('pythran==0.7.6.post1', 'console_scripts', 'pythran')()
  File "/home/nbecker/.local/lib/python3.5/site-packages/pythran-0.7.6.post1-py3.5.egg/pythran/run.py", line 126, in run
    **compile_flags(args))
  File "/home/nbecker/.local/lib/python3.5/site-packages/pythran-0.7.6.post1-py3.5.egg/pythran/toolchain.py", line 236, in compile_cxxfile
    extension_args = make_extension(**kwargs)
  File "/home/nbecker/.local/lib/python3.5/site-packages/pythran-0.7.6.post1-py3.5.egg/pythran/config.py", line 81, in make_extension
    numpy_blas = numpy_sys.get_info("blas")
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/system_info.py", line 372, in get_info
    return cl().get_info(notfound_action)
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/system_info.py", line 560, in get_info
    self.calc_info()
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/system_info.py", line 1679, in calc_info
    lib = self.has_cblas(info)
  File "/usr/lib64/python3.5/site-packages/numpy/distutils/system_info.py", line 1722, in has_cblas
    extra_postargs=info.get('extra_link_args', []))
  File "/usr/lib64/python3.5/distutils/ccompiler.py", line 734, in link_executable
    debug, extra_preargs, extra_postargs, None, target_lang)
  File "/usr/lib64/python3.5/distutils/unixccompiler.py", line 206, in link
    raise LinkError(msg)
distutils.errors.LinkError: Command "cc /tmp/tmppprhr2hh/tmp/tmppprhr2hh/source.o -L/usr/lib64 -lblas -o /tmp/tmppprhr2hh/a.out" failed with exit status 1

I see this is an issue with python3.5.  I was using
pythran -E mag_sqr.py
pythran3 mag_sqr.cpp (where pythran3 is the pythran script modified as
#!/usr/bin/python3)

problem is coming from config.py:
It's not just F24 numpy though, I tried anaconda version and get same result on py35:
import numpy.distutils.system_info as ns
In [16]: ns
Out[16]:
<module 'numpy.distutils.system_info' from '/home/nbecker/anaconda3/envs/py35/lib/python3.5/site-packages/numpy/distutils/system_info.py'>
In [17]: ns.get_info('blas')
/usr/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -lcblas

On Wed, Jul 20, 2016 at 05:58:10AM -0700, ndbecker wrote:

I see this is an issue with python3.5.  I was using
pythran -E mag_sqr.py
pythran3 mag_sqr.cpp (where pythran3 is the pythran script modified as
#!/usr/bin/python3)

Hi Neal and thanks for the report. I think it's a perfect opportunity to
try our beta native Python3 support, from branch feature/python-3. Can
you have a look ;-)

Thanks!
Same result (-lcblas).
For now, I disable blas check in config.py and hardcode:
# blas dependency
# numpy_blas = numpy_sys.get_info("blas")
# extension['libraries'].extend(numpy_blas.get('libraries', []))
# extension['library_dirs'].extend(numpy_blas.get('library_dirs', []))
extension['libraries'].extend(['openblas'])

On Wed, Jul 20, 2016 at 06:30:57AM -0700, ndbecker wrote:

Thanks!
Same result (-lcblas).
For now, I disable blas check in config.py and hardcode:
# blas dependency
# numpy_blas = numpy_sys.get_info("blas")
# extension['libraries'].extend(numpy_blas.get('libraries', []))
# extension['library_dirs'].extend(numpy_blas.get('library_dirs', []))
extension['libraries'].extend(['openblas'])

Then the best thing to do may be to add a blas entry in the config file.
It would override the numpy default if given...

@nbecker #593 makes it possible to customize the entry from the config file.

Fixed by  #593


JFXButton Raised color ?

Hey, I just downloaded and imported the library. I'm testing buttons but when I select raised button type there is no color option for it

Hi,
You need to change the background color using css style:
-fx-background-color: blue


Not show video image of .m3u8 when update to iOS 16.4

Bug

Platform

Which player are you experiencing the problem on:

iOS
Android
Windows UWP
Windows WPF

Environment info

React native info output:
None of these files exist:
  * node_modules/react-native/Libraries/Components/DatePicker/DatePickerIOS(.native|.native.js|.js|.native.jsx|.jsx|.native.json|.json|.native.ts|.ts|.native.tsx|.tsx|.native.svg|.svg)
  * node_modules/react-native/Libraries/Components/DatePicker/DatePickerIOS/index(.native|.native.js|.js|.native.jsx|.jsx|.native.json|.json|.native.ts|.ts|.native.tsx|.tsx|.native.svg|.svg)
  15 | import typeof ActivityIndicator from './Libraries/Components/ActivityIndicator/ActivityIndicator';
  16 | import typeof Button from './Libraries/Components/Button';
> 17 | import typeof DatePickerIOS from './Libraries/Components/DatePicker/DatePickerIOS';
     |                                   ^
  18 | import typeof DrawerLayoutAndroid from './Libraries/Components/DrawerAndroid/DrawerLayoutAndroid';
  19 | import typeof FlatList from './Libraries/Lists/FlatList';
  20 | import typeof Image from './Libraries/Image/Image';

Library version: 5.1.1
After update to iOS 16.4, I cannot build with the CLI. I have to build with Xcode, in device Rosetta mode (like Iphone 14 (Rosetta). When I go trought the app, I receive variou messagens like the bash, with problemas with DatePiCkerIOS, etc.
When I go to the video screen, if the video is mp4, for example, it goes ok. But when the video is .m3u8, the video not apperas (oly audio).
Anyone with his problem?

Bug
Platform
Which player are you experiencing the problem on:

iOS
Android
Windows UWP
Windows WPF

Environment info
React native info output:
None of these files exist:
  * node_modules/react-native/Libraries/Components/DatePicker/DatePickerIOS(.native|.native.js|.js|.native.jsx|.jsx|.native.json|.json|.native.ts|.ts|.native.tsx|.tsx|.native.svg|.svg)
  * node_modules/react-native/Libraries/Components/DatePicker/DatePickerIOS/index(.native|.native.js|.js|.native.jsx|.jsx|.native.json|.json|.native.ts|.ts|.native.tsx|.tsx|.native.svg|.svg)
  15 | import typeof ActivityIndicator from './Libraries/Components/ActivityIndicator/ActivityIndicator';
  16 | import typeof Button from './Libraries/Components/Button';
> 17 | import typeof DatePickerIOS from './Libraries/Components/DatePicker/DatePickerIOS';
     |                                   ^
  18 | import typeof DrawerLayoutAndroid from './Libraries/Components/DrawerAndroid/DrawerLayoutAndroid';
  19 | import typeof FlatList from './Libraries/Lists/FlatList';
  20 | import typeof Image from './Libraries/Image/Image';

Library version: 5.1.1
After update to iOS 16.4, I cannot build with the CLI. I have to build with Xcode, in device Rosetta mode (like Iphone 14 (Rosetta). When I go trought the app, I receive variou messagens like the bash, with problemas with DatePiCkerIOS, etc.
When I go to the video screen, if the video is mp4, for example, it goes ok. But when the video is .m3u8, the video not apperas (oly audio).
Anyone with his problem?

Aparentelly it occurs in simulators. When install in physical devices, it goes ok ...

facing same issue

Same for me..

source={{
uri: 'http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8',
type: 'm3u8' }
}

Same issue here when loading a m3u8 file in the video player, only since upgrading to iOS 16.4.

Does anyone tested on V6 version?

@freeboub Yes I'm on 6.0.0-alpha.6 and the issue exists.

Confirming this on ios simulator v16.4, and it works on a real device (alpha6)

Same here, I should have mentioned that it only occurs for me on iOS simulator as well, a physical iOS device works fine.

Happening here too on both versions 5 and 6 alpha

Audio works but video doesn't display. Nothing in the logs, error callbacks not called. If I step through the swift code then it reaches the end and calls handlePlayerItemDidReachEnd successfully.
I'm not familiar enough with the code to understand the specifics of what's going on with AVPlayer here unfortunately.

I also have such a problem

I had the same video playback issue on latest v16.4 iOS simulator.
Temp fix was to install old simulators from v16 via XCode Preferences > Platforms Tab (formerly Components)

I confirm than I had the same issue:

Black video but audio working
Appears only on the simulator (currently on iOS 16.4)

Building the app on physical device seems to be OK!

Issue with simulator. Let's close this issue

@freeboub the issue still exists. I understand that it has lower priority if it only occurs in simulators and not on real devices in production, but this developer experience is far away from being great. There is no general problem with AVPlayer on iOS 16.4, so I'm guessing this bug is related to react-native-video. I kindly ask to open the ticket again.

@michaelknoch I think based on this thread https://forums.developer.apple.com/forums/thread/727288?page=2 it can clearly be stated that was some system bug on Apple's side. Which is now fixed in latest Xcode/Simulator versions.
cc @freeboub


强化学习之后生成不完整的描述？

After reinforcement learning, the description will be incomplete such as:
a motorcycle parked in a parking lot with a ..

确实是会这样的。你看一下bad ending rate，大概多大。如果你用的这个库训练的话，这个rate不会特别大。原因是cider metric的问题。如果你比如说加一个bad ending penalty reward，应该能alleviate这个问题我觉得。Ruotian LuoOn Dec 18, 2023, at 9:26 PM, Xiong-can @.***> wrote:
After reinforcement learning, the description will be incomplete such as:
a motorcycle parked in a parking lot with a ..
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

确实是会这样的。你看一下bad ending rate，大概多大。如果你用的这个库训练的话，这个rate不会特别大。原因是cider metric的问题。如果你比如说加一个bad ending penalty reward，应该能alleviate这个问题我觉得。Ruotian LuoOn Dec 18, 2023, at 9:26 PM, Xiong-can @.> wrote: After reinforcement learning, the description will be incomplete such as: a motorcycle parked in a parking lot with a .. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.>

bad ending rate大概是1/2，如何添加bad endiing penalty reward才能alleviate这个问题呢？能请我一下您的模型的这个bad ending rate为多少吗？谢谢！

这个不太对。scst原本的paper会在算cider的时候加上eos token（原本的cider不加）。我之前试的话如果不加eos会有1/3的bad ending。你有follow这样的cider计算方式吗？Ruotian LuoOn Dec 18, 2023, at 10:15 PM, Xiong-can @.***> wrote:
确实是会这样的。你看一下bad ending rate，大概多大。如果你用的这个库训练的话，这个rate不会特别大。原因是cider metric的问题。如果你比如说加一个bad ending penalty reward，应该能alleviate这个问题我觉得。Ruotian LuoOn Dec 18, 2023, at 9:26 PM, Xiong-can @.> wrote: After reinforcement learning, the description will be incomplete such as: a motorcycle parked in a parking lot with a .. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.>
bad ending rate大概是1/2，如何添加bad endiing penalty reward才能alleviate这个问题呢？能请我一下您的模型的这个bad ending rate为多少吗？谢谢！
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

原本的cider不加）。我之前试的话如果不加eos会有1/3的bad ending。你有follow这样的cider计算方式吗？

我的cider计算方式是M2(meshed memory)论文中的evaluation方法计算cider，强化学习计算分数的时候应该是添加了eos进行计算的，这个是我的强化学习部分主要的代码
Rewards
        caps_gen = text_field.decode(out.view(-1, seq_len))
        caps_gt = list(itertools.chain(*([c, ] * beam_size for c in data["text"])))
        caps_gen, caps_gt = tokenizer_pool.map(evaluation.PTBTokenizer.tokenize, [caps_gen, caps_gt])
        reward = cider.compute_score(caps_gt, caps_gen)[1].astype(np.float32)
        reward = torch.from_numpy(reward).to(device).view(detections.shape[0], beam_size)
        reward_baseline = torch.mean(reward, -1, keepdim=True)
        loss = -torch.mean(log_prob, -1) * (reward - reward_baseline)
        loss = loss.mean()

这个是cider分数的计算方式：
def compute_cider(self):
def counts2vec(cnts):
"""
Function maps counts of ngram to vector of tfidf weights.
The function returns vec, an array of dictionary that store mapping of n-gram and tf-idf weights.
The n-th entry of array denotes length of n-grams.
:param cnts:
:return: vec (array of dict), norm (array of float), length (int)
"""
vec = [defaultdict(float) for _ in range(self.n)]
length = 0
norm = [0.0 for _ in range(self.n)]
for (ngram,term_freq) in cnts.items():
# give word count 1 if it doesn't appear in reference corpus
df = np.log(max(1.0, self.doc_frequency[ngram]))
# ngram index
n = len(ngram)-1
# tf (term_freq) * idf (precomputed idf) for n-grams
vec[n][ngram] = float(term_freq)*(self.ref_len - df)
# compute norm for the vector.  the norm will be used for computing similarity
norm[n] += pow(vec[n][ngram], 2)
            if n == 1:
                length += term_freq
        norm = [np.sqrt(n) for n in norm]
        return vec, norm, length

    def sim(vec_hyp, vec_ref, norm_hyp, norm_ref, length_hyp, length_ref):
        '''
        Compute the cosine similarity of two vectors.
        :param vec_hyp: array of dictionary for vector corresponding to hypothesis
        :param vec_ref: array of dictionary for vector corresponding to reference
        :param norm_hyp: array of float for vector corresponding to hypothesis
        :param norm_ref: array of float for vector corresponding to reference
        :param length_hyp: int containing length of hypothesis
        :param length_ref: int containing length of reference
        :return: array of score for each n-grams cosine similarity
        '''
        delta = float(length_hyp - length_ref)
        # measure consine similarity
        val = np.array([0.0 for _ in range(self.n)])
        for n in range(self.n):
            # ngram
            for (ngram,count) in vec_hyp[n].items():
                # vrama91 : added clipping
                val[n] += min(vec_hyp[n][ngram], vec_ref[n][ngram]) * vec_ref[n][ngram]

            if (norm_hyp[n] != 0) and (norm_ref[n] != 0):
                val[n] /= (norm_hyp[n]*norm_ref[n])

            assert(not math.isnan(val[n]))
            # vrama91: added a length based gaussian penalty
            val[n] *= np.e**(-(delta**2)/(2*self.sigma**2))
        return val

    scores = []
    for test, refs in zip(self.ctest, self.crefs):
        # compute vector for test captions
        vec, norm, length = counts2vec(test)
        # compute vector for ref captions
        score = np.array([0.0 for _ in range(self.n)])
        for ref in refs:
            vec_ref, norm_ref, length_ref = counts2vec(ref)
            score += sim(vec, vec_ref, norm, norm_ref, length, length_ref)
        # change by vrama91 - mean of ngram scores, instead of sum
        score_avg = np.mean(score)
        # divide by number of references
        score_avg /= len(refs)
        # multiply score by 10
        score_avg *= 10.0
        # append score of an image to the score list
        scores.append(score_avg)
    return scores

def compute_score(self):
    # compute cider score
    score = self.compute_cider()
    # debug
    # print score
    return np.mean(np.array(score)), np.array(score)

m2的是有问题的。我1/3的结果就是m2跑出来的。我记得m2是没有加eos的。Ruotian LuoOn Dec 19, 2023, at 10:25 AM, Xiong-can @.***> wrote:
原本的cider不加）。我之前试的话如果不加eos会有1/3的bad ending。你有follow这样的cider计算方式吗？
我的cider计算方式是M2(meshed memory)论文中的evaluation方法计算cider，强化学习计算分数的时候应该是添加了eos进行计算的，这个是我的强化学习部分主要的代码
Rewards
caps_gen = text_field.decode(out.view(-1, seq_len))
caps_gt = list(itertools.chain(*([c, ] * beam_size for c in data["text"])))
caps_gen, caps_gt = tokenizer_pool.map(evaluation.PTBTokenizer.tokenize, [caps_gen, caps_gt])
reward = cider.compute_score(caps_gt, caps_gen)[1].astype(np.float32)
reward = torch.from_numpy(reward).to(device).view(detections.shape[0], beam_size)
reward_baseline = torch.mean(reward, -1, keepdim=True)
loss = -torch.mean(log_prob, -1) * (reward - reward_baseline)
loss = loss.mean()
这个是cider分数的计算方式：
def compute_cider(self):
def counts2vec(cnts):
"""
Function maps counts of ngram to vector of tfidf weights.
The function returns vec, an array of dictionary that store mapping of n-gram and tf-idf weights.
The n-th entry of array denotes length of n-grams.
:param cnts:
:return: vec (array of dict), norm (array of float), length (int)
"""
vec = [defaultdict(float) for _ in range(self.n)]
length = 0
norm = [0.0 for _ in range(self.n)]
for (ngram,term_freq) in cnts.items():
give word count 1 if it doesn't appear in reference corpus
df = np.log(max(1.0, self.doc_frequency[ngram]))
ngram index
n = len(ngram)-1
tf (term_freq) * idf (precomputed idf) for n-grams
vec[n][ngram] = float(term_freq)*(self.ref_len - df)
compute norm for the vector.  the norm will be used for computing similarity
norm[n] += pow(vec[n][ngram], 2)
if n == 1:
length += term_freq
norm = [np.sqrt(n) for n in norm]
return vec, norm, length
def sim(vec_hyp, vec_ref, norm_hyp, norm_ref, length_hyp, length_ref):
    '''
    Compute the cosine similarity of two vectors.
    :param vec_hyp: array of dictionary for vector corresponding to hypothesis
    :param vec_ref: array of dictionary for vector corresponding to reference
    :param norm_hyp: array of float for vector corresponding to hypothesis
    :param norm_ref: array of float for vector corresponding to reference
    :param length_hyp: int containing length of hypothesis
    :param length_ref: int containing length of reference
    :return: array of score for each n-grams cosine similarity
    '''
    delta = float(length_hyp - length_ref)
    # measure consine similarity
    val = np.array([0.0 for _ in range(self.n)])
    for n in range(self.n):
        # ngram
        for (ngram,count) in vec_hyp[n].items():
            # vrama91 : added clipping
            val[n] += min(vec_hyp[n][ngram], vec_ref[n][ngram]) * vec_ref[n][ngram]

        if (norm_hyp[n] != 0) and (norm_ref[n] != 0):
            val[n] /= (norm_hyp[n]*norm_ref[n])

        assert(not math.isnan(val[n]))
        # vrama91: added a length based gaussian penalty
        val[n] *= np.e**(-(delta**2)/(2*self.sigma**2))
    return val

scores = []
for test, refs in zip(self.ctest, self.crefs):
    # compute vector for test captions
    vec, norm, length = counts2vec(test)
    # compute vector for ref captions
    score = np.array([0.0 for _ in range(self.n)])
    for ref in refs:
        vec_ref, norm_ref, length_ref = counts2vec(ref)
        score += sim(vec, vec_ref, norm, norm_ref, length, length_ref)
    # change by vrama91 - mean of ngram scores, instead of sum
    score_avg = np.mean(score)
    # divide by number of references
    score_avg /= len(refs)
    # multiply score by 10
    score_avg *= 10.0
    # append score of an image to the score list
    scores.append(score_avg)
return scores

def compute_score(self):
# compute cider score
score = self.compute_cider()
# debug
# print score
return np.mean(np.array(score)), np.array(score)
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

m2的是有问题的。我1/3的结果就是m2跑出来的。我记得m2是没有加eos的。Ruotian LuoOn Dec 19, 2023, at 10:25 AM, Xiong-can @.> wrote: 原本的cider不加）。我之前试的话如果不加eos会有1/3的bad ending。你有follow这样的cider计算方式吗？ 我的cider计算方式是M2(meshed memory)论文中的evaluation方法计算cider，强化学习计算分数的时候应该是添加了eos进行计算的，这个是我的强化学习部分主要的代码 Rewards caps_gen = text_field.decode(out.view(-1, seq_len)) caps_gt = list(itertools.chain(([c, ] * beam_size for c in data["text"]))) caps_gen, caps_gt = tokenizer_pool.map(evaluation.PTBTokenizer.tokenize, [caps_gen, caps_gt]) reward = cider.compute_score(caps_gt, caps_gen)[1].astype(np.float32) reward = torch.from_numpy(reward).to(device).view(detections.shape[0], beam_size) reward_baseline = torch.mean(reward, -1, keepdim=True) loss = -torch.mean(log_prob, -1) * (reward - reward_baseline) loss = loss.mean() 这个是cider分数的计算方式： def compute_cider(self): def counts2vec(cnts): """ Function maps counts of ngram to vector of tfidf weights. The function returns vec, an array of dictionary that store mapping of n-gram and tf-idf weights. The n-th entry of array denotes length of n-grams. :param cnts: :return: vec (array of dict), norm (array of float), length (int) """ vec = [defaultdict(float) for _ in range(self.n)] length = 0 norm = [0.0 for _ in range(self.n)] for (ngram,term_freq) in cnts.items(): # give word count 1 if it doesn't appear in reference corpus df = np.log(max(1.0, self.doc_frequency[ngram])) # ngram index n = len(ngram)-1 # tf (term_freq) * idf (precomputed idf) for n-grams vec[n][ngram] = float(term_freq)(self.ref_len - df) # compute norm for the vector. the norm will be used for computing similarity norm[n] += pow(vec[n][ngram], 2) if n == 1: length += term_freq norm = [np.sqrt(n) for n in norm] return vec, norm, length def sim(vec_hyp, vec_ref, norm_hyp, norm_ref, length_hyp, length_ref): ''' Compute the cosine similarity of two vectors. :param vec_hyp: array of dictionary for vector corresponding to hypothesis :param vec_ref: array of dictionary for vector corresponding to reference :param norm_hyp: array of float for vector corresponding to hypothesis :param norm_ref: array of float for vector corresponding to reference :param length_hyp: int containing length of hypothesis :param length_ref: int containing length of reference :return: array of score for each n-grams cosine similarity ''' delta = float(length_hyp - length_ref) # measure consine similarity val = np.array([0.0 for _ in range(self.n)]) for n in range(self.n): # ngram for (ngram,count) in vec_hyp[n].items(): # vrama91 : added clipping val[n] += min(vec_hyp[n][ngram], vec_ref[n][ngram]) * vec_ref[n][ngram] if (norm_hyp[n] != 0) and (norm_ref[n] != 0): val[n] /= (norm_hyp[n]norm_ref[n]) assert(not math.isnan(val[n])) # vrama91: added a length based gaussian penalty val[n] = np.e(-(delta2)/(2*self.sigma2)) return val scores = [] for test, refs in zip(self.ctest, self.crefs): # compute vector for test captions vec, norm, length = counts2vec(test) # compute vector for ref captions score = np.array([0.0 for _ in range(self.n)]) for ref in refs: vec_ref, norm_ref, length_ref = counts2vec(ref) score += sim(vec, vec_ref, norm, norm_ref, length, length_ref) # change by vrama91 - mean of ngram scores, instead of sum score_avg = np.mean(score) # divide by number of references score_avg /= len(refs) # multiply score by 10 score_avg = 10.0 # append score of an image to the score list scores.append(score_avg) return scores def compute_score(self): # compute cider score score = self.compute_cider() # debug # print score return np.mean(np.array(score)), np.array(score) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.*>

那我应该如何alleviate这个问题？我是应该增加一个bad endiing penalty reward？还是修改一下cider的计算方式？是不是Cider-D的计算方式就是增加了一个惩罚因子？

1我没试过reward，但我觉得试一试挺意思的。2 你可以修改m2里面cider的计算方式，加入eos（注意预处理的时候需要加eos，算的时候也要加）3 另一个你可以加上xe loss，blaance一下Ruotian Luo在 2023年12月20日，上午10:49，Xiong-can @.***> 写道：
m2的是有问题的。我1/3的结果就是m2跑出来的。我记得m2是没有加eos的。Ruotian LuoOn Dec 19, 2023, at 10:25 AM, Xiong-can @.> wrote: 原本的cider不加）。我之前试的话如果不加eos会有1/3的bad ending。你有follow这样的cider计算方式吗？ 我的cider计算方式是M2(meshed memory)论文中的evaluation方法计算cider，强化学习计算分数的时候应该是添加了eos进行计算的，这个是我的强化学习部分主要的代码 Rewards caps_gen = text_field.decode(out.view(-1, seq_len)) caps_gt = list(itertools.chain(([c, ] * beam_size for c in data["text"]))) caps_gen, caps_gt = tokenizer_pool.map(evaluation.PTBTokenizer.tokenize, [caps_gen, caps_gt]) reward = cider.compute_score(caps_gt, caps_gen)[1].astype(np.float32) reward = torch.from_numpy(reward).to(device).view(detections.shape[0], beam_size) reward_baseline = torch.mean(reward, -1, keepdim=True) loss = -torch.mean(log_prob, -1) * (reward - reward_baseline) loss = loss.mean() 这个是cider分数的计算方式： def compute_cider(self): def counts2vec(cnts): """ Function maps counts of ngram to vector of tfidf weights. The function returns vec, an array of dictionary that store mapping of n-gram and tf-idf weights. The n-th entry of array denotes length of n-grams. :param cnts: :return: vec (array of dict), norm (array of float), length (int) """ vec = [defaultdict(float) for _ in range(self.n)] length = 0 norm = [0.0 for _ in range(self.n)] for (ngram,term_freq) in cnts.items(): # give word count 1 if it doesn't appear in reference corpus df = np.log(max(1.0, self.doc_frequency[ngram])) # ngram index n = len(ngram)-1 # tf (term_freq) * idf (precomputed idf) for n-grams vec[n][ngram] = float(term_freq)(self.ref_len - df) # compute norm for the vector. the norm will be used for computing similarity norm[n] += pow(vec[n][ngram], 2) if n == 1: length += term_freq norm = [np.sqrt(n) for n in norm] return vec, norm, length def sim(vec_hyp, vec_ref, norm_hyp, norm_ref, length_hyp, length_ref): ''' Compute the cosine similarity of two vectors. :param vec_hyp: array of dictionary for vector corresponding to hypothesis :param vec_ref: array of dictionary for vector corresponding to reference :param norm_hyp: array of float for vector corresponding to hypothesis :param norm_ref: array of float for vector corresponding to reference :param length_hyp: int containing length of hypothesis :param length_ref: int containing length of reference :return: array of score for each n-grams cosine similarity ''' delta = float(length_hyp - length_ref) # measure consine similarity val = np.array([0.0 for _ in range(self.n)]) for n in range(self.n): # ngram for (ngram,count) in vec_hyp[n].items(): # vrama91 : added clipping val[n] += min(vec_hyp[n][ngram], vec_ref[n][ngram]) * vec_ref[n][ngram] if (norm_hyp[n] != 0) and (norm_ref[n] != 0): val[n] /= (norm_hyp[n]norm_ref[n]) assert(not math.isnan(val[n])) # vrama91: added a length based gaussian penalty val[n] = np.e(-(delta2)/(2self.sigma2)) return val scores = [] for test, refs in zip(self.ctest, self.crefs): # compute vector for test captions vec, norm, length = counts2vec(test) # compute vector for ref captions score = np.array([0.0 for _ in range(self.n)]) for ref in refs: vec_ref, norm_ref, length_ref = counts2vec(ref) score += sim(vec, vec_ref, norm, norm_ref, length, length_ref) # change by vrama91 - mean of ngram scores, instead of sum score_avg = np.mean(score) # divide by number of references score_avg /= len(refs) # multiply score by 10 score_avg = 10.0 # append score of an image to the score list scores.append(score_avg) return scores def compute_score(self): # compute cider score score = self.compute_cider() # debug # print score return np.mean(np.array(score)), np.array(score) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.>
那我应该如何alleviate这个问题？我是应该增加一个bad endiing penalty reward？还是修改一下cider的计算方式？是不是Cider-D的计算方式就是增加了一个惩罚因子？
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>


Issue with offline playlists

Hey,
Having an issue with offline playlists.
After creating a session and logging in for the first time, offline playlists download just fine. But after logging out and logging back in again, a playlist's offlineStatus property is immediately set to SP_PLAYLIST_OFFLINE_STATUS_YES as soon as its markedForOfflinePlayback property is set to YES. Each of the playlist's tracks offlineStatus property is set to SP_TRACK_OFFLINE_WAITING and the session objects's offlineSyncing property has the value NO. The offlinePlaylistsRemaining and offlineTracksRemaning properties of the session object are both zero. The connectionState property of the the session object has the value SP_CONNECTION_STATE_LOGGED_IN.
Any ideas?

Heh, interesting that we both created the same issue at the same time.


chore: automated PR to update ReleaseNotes for 202402.07.0 VHD

This is an automated PR to bump ReleaseNotes for the VHD release with image version 202402.07.0

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.313%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2285

Relevant Lines:
2881

💛  - Coveralls


#36 | BUG/3 Suggestions and original sentences are loaded simultaneously | Artem

Description
Suggestions and original sentences are loaded simultaneously.
The changes ensure that the original sentence and suggestions are initially set to "Loading..." and only change when the suggestions are fully loaded from the API.
Changes:

Defined a new function getSuggestionsFromApi at the beginning of the Home function. This function is responsible for fetching suggestions from the API, updating the suggestions state, setting the original sentence, and updating the loading state.

Modified the loadRandomSentenceFromFile function inside the first useEffect hook to call getSuggestionsFromApi after fetching the random sentence. This ensures that the original sentence is set only after the suggestions have been loaded.

Modified the second useEffect hook to call getSuggestionsFromApi only when randomText is not "Loading...". This ensures that the suggestions are fetched and the loading state is updated only when the original sentence is not in the loading state.

Related to
Bug fix #36
Checklist:

[x] My code follows the style guidelines of this project
[x] I have carefully reviewed my own code
[x] I have commented my code
[x] I have updated any documentation

Hi, do note there's a conflict between this and the loading page fix code. Also note the loading page fix did already solve this partially, so make sure this piece of code is actually still required or not

Going to close this as I believe it's not needed anymore with the spinner solution we have right now. Feel free to re-open if you still believe this is useful


"ShopSmart" text

The topbar takes too much space and i think we have two approaches to solve it.

A Creative Idea about the topbar.
Simply reduce the space.

If anyone want to proceed with the first one then he/she must provide a brief and will be also prioritized for this only issue.

Or we can just reduce the extra space taken by the Top-App bar.

@yuvrajsinghgmx Can i implement auto hiding top app bar on scrolling? If yes the please assign me the issue.

I also noticed it and would be interested in working on it. The top bar is currently simple. We can enhance it by adding a search button for users to search for items and update it when a user selects items. When the user selects items, the top bar changes to show the number of selected items and the delete button. We can implement something similar to this.

We can introduce this.

Ok then please assign it to me, and I'll work on it.

#54 is working on the search feature so please keep this in mind.
I am assigning this to you.

Ok, I'll talk to him, there won't be any conflict.

I apologize for being late due to my midterms.
I encountered some issues while improving the top-app bar. Changing the height of the top bar looked so straight but it wasn't that easy. I tried a lot, but couldn't figure it out. It will take more time than I thought I guess.
Also, we don't get the same default color in the top bars as it is implemented in the Smart-Bottom navigation bar, so I left it as default for now.
I am making a PR please take a look at it.


Possible Deadlock in BehaviorSubject.Subscribe

Hello and thank you for using dotnet/reactive. Please select a category and detail your issue by answering the questions there:
Bug
We recently had a deadlock in our application, when while the Subscribe was active, the UnSubscribe (via the Dispose) was called on a differen thread and this caused a deadlock. This happened in our case as the subscription was cancelled before the Subscribe was even called (due to a quickly opening and closing of a screen).
We found the issue to be in the following file: https://github.com/dotnet/reactive/blob/main/Rx.NET/Source/src/System.Reactive/Subjects/BehaviorSubject.cs
I suggest to change the following code from
    public override IDisposable Subscribe(IObserver<T> observer)
        {
            if (observer == null)
            {
                throw new ArgumentNullException(nameof(observer));
            }

            Exception? ex;

            lock (_gate)
            {
                CheckDisposed();

                if (!_isStopped)
                {
                    _observers = _observers.Add(observer);
                    observer.OnNext(_value);
                    return new Subscription(this, observer);
                }

                ex = _exception;
            }

            if (ex != null)
            {
                observer.OnError(ex);
            }
            else
            {
                observer.OnCompleted();
            }

            return Disposable.Empty;
        }

To:
    public override IDisposable Subscribe(IObserver<T> observer)
    {
        if (observer == null)
        {
            throw new ArgumentNullException(nameof(observer));
        }

        Exception? ex;

        lock (this._gate)
        {
            this.CheckDisposed();

            this._observers = this._observers.Add(observer);
            ex = this._exception;
        }

        // Note: This first IF statement with the observer.OnNext was originally inside the lock
        // This could cause a deadlock in certain circumstances. As part of the fix we have made a copy of the
        // BehaviorSubject from Reactive.Net and moved this outside the lock to prevent a deadlock
        // when a subject is being disposed while being inside the Subscribe.
        // See also: https://dev.azure.com/bronkhorst/IT/_wiki/wikis/Software-development.wiki/627/043-deadlock-terug-navigeren-Android
        if (!this._isStopped)
        {
            observer.OnNext(this._value);
            return new Subscription(this, observer);
        }
        else if (ex != null)
        {
            observer.OnError(ex);
        }
        else
        {
            observer.OnCompleted();
        }

        return Disposable.Empty;
    }

The main change is that the IObserver.OnNext call is outside the lock. This also matches what happens when BehaviourSubject.OnNext from the is called, where the IObserver.OnNext is also outside the lock. This will prevent future deadlock scenarios.

Which library version?
6.0.0
What are the platform(s), environment(s) and related component version(s)?
Android/IOS/Windows (probably all platforms, but verified on these)

What is the use case or problem?
A deadlock can occur.

What is the expected outcome?
No deadlock possibility.

What is the actual outcome?
In our case a deadlock.

What is the stacktrace of the exception(s) if any?
There is no exception.

Do you have a code snippet or project that reproduces the problem?

BehaviorSubject is very special in that it promises to emit the stored value to the incoming observer before it relays future values. The trouble is, a concurrent BehaviorSubject::OnNext call can happen while a BehaviourSubject::Subscribe is ongoing. OnNext mustn't be called concurrently so the two method calls must be serialized, hence the emission under the lock.

This happened in our case as the subscription was cancelled before the Subscribe was even called

The IDisposable returned by the BehaviorSubject::Subscribe can't be disposed upfront as it does not exist until the method returns it. There must have been something else causing the deadlock. Could you please share more details about the relevant code of yours?

Thank you for your quick reply, it's much appreciated.
Thinking about it, I agree that the Dispose couldn't cause the deadlock, except for when a subscribe causes a dispose of a previous subscription (but why should it?). I must have been confused as several dispose calls were waiting to get the lock and in the stacktrace that was always as a result of an Subscribe call. This was on Android though which may have caused issues in the stacktrace.
To be honest, there were a bunch of issues happening at the same time divided over several threads (up to 15 according to the parallel stacks window) which caused problems, making it difficult to separate which calls exactly caused the deadlock. (It took me a week and half to figure it all out)
I do know that the dispose was involved and making this change solved the deadlock for us. :)
As for what emits when; if the promise that the stored value is first emitted before any OnNext calls is sacred, then I agree my suggested change can't be made, even though it is unlikely an incoming OnNext call is emiting before the Subscribe completes. But if you believe it can happen (and I do think it can in theory), then by extension, there is no guarantee that all OnNext calls are emitted in the order they are received, which sort of makes the promise a bit of a weird one to make (what is the point of it).
I just wanted to put my issue and solution out here, as for us it works (and it may help other people). I do think it's clear a deadlock "could" happen there. But if my solution doesn't fit in the context of what the BehaviorSubject promises (or should promise) then feel free to close this issue.
Once again, I appreciate your quick response. Thanks again.


Top issues sometimes appears empty, sometimes has wrong empty state

Sometimes appears empty:

In this case we have the slow endpoint issue, which isn't represented here:

Expected behavior - show empty state and not an empty title.

Current empty state is wrong - @ionuiux please revise. Title should be- 'No issues found'  and text should be  No issues , 'No issues found in this asset`. The icon should also not be an X but a positive icon.

Design mockup - https://www.figma.com/design/MxA95ZHK5NqGX4ptFQljRJ/⭐-Design-System-3.0?node-id=5055-54603&node-type=frame&t=FjKTfhYL1pOlAv80-11


mraa does not work any more with Kernel 5.X  : legacyGPIO REMOVED from kernel

I am successfully compiling mraa on a rasppberryPi4-64  with debian buster image kernel v5.4.
Unfortunately mrra is not operational because legacy GPIO is not constructed any more and a bunch of files is missing in /sys/ folder.
I might be a little ahead on that move but this inconvenience may spread over other users soon. Are there plans to bring mraa to use regular GPIO definitions. and make it available with recent kernels. I
If anybody knows a reversible trick tu build the legacy GPIO  please let me know

@g-vidal thanks for being a long time supporter of the project. A new milestone that I see relevant and needs to be promoted more is the need to implement the io_uring APIs that were made available with more recent kernels. This will keep the project relevant and modern so I hope it addresses at least part of your concerns.

@g-vidal We have added support for gpio chardev long ago in MRAA. That'll work with recent kernels. Please check 96boards platform support code where you'll get some reference. For switching over, you just need to provide a flag and an array of gpio chardev mapping. Finally MRAA will try to use chardev or legacy gpio depending on the kernel revision.
Let me know if you need any help.

Many thanks @Propanu  and @Mani-Sadhasivam I am very busy to support teachers during the covid-19 containment with distance learning. I am deeply overbooked at the moment but I'll check ASAP and provide feedback.


how to install through yum command?

any idea about simplified installation steps, add repository to use yum install/upgrade command?

We had some investigation about adding packet-agent to public repository before. The conclusion is it's hard for us to do that. Some communities use the rule "Don't call us, we will call you", which means they will collect the packages by themselves. Some set high standard to approve the merge request, Fedora for example: https://fedoraproject.org/wiki/Join_the_package_collection_maintainers

any idea about a customized repo to be added into yum command like https://www.thegeekdiary.com/centos-rhel-how-to-create-and-host-yum-repository-over-httpd/

We investigated and talked about the customized repo before. We thought it will not simplify the steps for users, as they need to import and refresh repo first and then use yum command to install packet agent. Also, hosting the repo will increase the burden of mantaining servers and services.


Can't install this node with a "standard" setup for pi (Node v14 is installed)

I've installed Node-RED using the bash script provided by the official site, but this installs node v14 (in the docs still says v12), and nothing other version.
But when i tried to install this node using the Web UI it failed and the logs said it needs node v16. I'm afraid to update manually node to v16 and break something (i'm an expert in braking things already stable in my Pi). If the official script requires specifically v14 (maybe due some quirk in Pi 3b+, i dont know).
Is there any option left for me?

I open the raw for the script and there is a option in it for installation of node-red with node js 16 (https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

This is the reason beacuse you have node 14


META-ISSUE: Performance task force

Performance discussion in a environment such as libModSecurity which is deployed in different manners and meant to be used in different scenarios is always interesting because it leads to circumstances where the benefit for some may be negative for others. That said, this ticket will be open forever :).

The definition of performance  optimization is
The process of making something, especially a computer system, work as effectively as possible.

Due the fact that some optimization may lead to the benefit of a few, different angles and point of views are expected to be discussed. Luckily for us numbers are the numbers to everybody. :)
The main point is: what is a good or bad performance?
Usually performance is a trade-of between: CPU, Memory, and I/O in a triangle shape. Decrease the usage of one, increase the usage of other. libModSecurity was designed to support low latency; Even if it costs throughput. That is kind of desirable in o cloud-like deployment.
How to measure the performance
Usually ModSecurity is deployed together with different components:

libModSecurity
Connetor: ModSecurity-nginx, ModSecurity-apache [etc].
The webserver:  nginx, apache [etc].

Don't be confused, the numbers here are about the library ModSecurity. The library already contains the benchmark utility and the stap scripts necessary to plot the data.

https://github.com/SpiderLabs/ModSecurity/tree/v3/master/test/benchmark
https://github.com/SpiderLabs/ModSecurity-nginx/blob/master/ngx-modsec.stp

Other testing components, tools and methods may be suitable for their own tickets and discussion.
What the numbers means
Are "bad" benchmark numbers means that my web application will be that bad? Not necessarily. Although it can have a relation. In a real world deployment other things like: latency and our back end application will add substantial processing time, leading the overhead of the WAF to be negligible, at least in a theoretical scenario were: well performant rules met performant core.
Notice that the request also play a important role in this equation, hard to choose what it a typical one these days.
Knowing all those variables, the numbers gives us a strong indication where to look in order to improve the performance.
How can I send a suggestion here?
The idea is taking everybody's input into consideration. The uptake on the decision is what is best for the community overall, so, in order to everybody understand your performance problem please explain your suggestion pointing to facts.
I want to participate but I'm not following the discussions
There are a few tasks that you may want to help:
Variable computation up to utilization
ModSecurity v3 computes all variables regardless if it is being used or not, the variable are gradativelly filled everytime the a new piece of information is delivered to ModSecurity. This does not need to be that way. As the rules are pre-compiled, the variable just needs to be computed (and therefore allocate memory for it) if they are really used. The architecture already fits this implementation, it just needs to be done. In this process patches thatremove some configuration directives that will be deprecated will be more than welcome. As the example of: SecResponseBodyAccess.
Memory pools to avoid memory fragmentation
Memory fragmentation in our case is a consequence to very little pieces of informations that are allocated in each transaction. In busy servers that most certainly will be an issue. Further info: Memory fragmentation
Technical investigation on the feasibility to reduce the unused variables or variables with same or similar content.
The are some variables that may hold the same content as the example of:
https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#REQUEST_LINE
https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#REQUEST_URI
https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#REQUEST_URI_RAW
Do they need to co-exist? If so, can we use offsets in a way to use the same memory space to represent it?
Support for cloud-like collection storage.
Collections are now saved into a key-pair storage which has the benefit of simplicity but not optimal in terms of performance. Handle that to an external process will lead to a good reduction of CPU usage.
Performant implementation for popular transformations.
Some of the transformations are executed too many times, some already does in-memory operations, but better logic in the implementation will help a lot. Assembly is welcomed here.
[to be update with new stuff as needed.]

References
Requests per Second: Realworld scenario: https://sourceforge.net/p/mod-security/mailman/message/36286327/
Requests per Second: Localhost: https://sourceforge.net/p/mod-security/mailman/message/36285879/
Substantial Slowdown report : https://sourceforge.net/p/mod-security/mailman/message/36285471/
Performance Expectations : https://sourceforge.net/p/mod-security/mailman/message/36285446/
Limitations in core code and TODOs : https://sourceforge.net/p/mod-security/mailman/message/36285367/
Problem with Rule::evaluate : https://sourceforge.net/p/mod-security/mailman/message/36284403/

I do not understand the "designed to support low latency"? What does this mean exactly? I take it, this is not the latency that the client sees, or is it?
Other than that, I get the feeling these are all very good considerations, but we are not yet in a situation where libModSecurity has efficient code and further optimization demands priorities in the classic triangle. We are in a situation where inefficient code means subpar performance and I do not see this mirrored above. Various people who have taken a closer look at the source code and they have pointed out where they see room for optimization. There is only a small intersection with the ideas described in the introduction to this issue. We miss a statement of the lead developer with regards to these suggestions.

REQUEST_LINE vs. REQUEST_URI vs. REQUEST_URI_RAW vs. REQUEST_FILENAME
Yes, they are all needed in different situations. If you can "use offsets in a way to use the same memory space to represent " them, then yes, by all means. But these 3 cousins are really important.
 Figured I would throw some of our results and configuration information out there for yah:
We run NGINX+ OpenResty + Kong(API Gateway application) for REST/SOAP/Websocket API services.
2020/02/04 06:53:02 [debug] Kong: 1.4.3
2020/02/04 06:48:58 [debug] ngx_lua: 10015
2020/02/04 06:48:58 [debug] nginx: 1015008
2020/02/04 06:48:58 [debug] Lua: LuaJIT 2.1.0-beta3

6 worker processes.

SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000

SecRule TX:/^MSC_/ "!@streq 0" \
        "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"

SecResponseBodyAccess Off
SecResponseBodyMimeType text/plain text/html text/xml
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial

SecTmpDir /tmp/

Dir /tmp/

#SecUploadDir /opt/modsecurity/var/upload/
#SecUploadKeepFiles RelevantOnly
#SecUploadFileMode 0600
#SecDebugLog /opt/modsecurity/var/log/debug.log
#SecDebugLogLevel 3
SecAuditEngine Off
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Concurrent
SecAuditLog /usr/local/kong/logs/modsec_audit.log
# Specify the path for concurrent audit logging.
#SecAuditLogStorageDir /opt/modsecurity/var/audit/
SecArgumentSeparator &
SecCookieFormat 0
SecUnicodeMapFile unicode.mapping 20127
SecStatusEngine Off

LoadTest environment:
Beefy machine running WRK client, parameters:

TEST_DURATION (seconds) | 300
TEST_PAYLOAD_SIZE (kb) | 10
TEST_THREADS | 10
TEST_CONNECTIONS | 10

RESULTS
Before installing the WAF:

THROUGHPUT | 2185 TPS
BYTES |<PHONE_NUMBER>2
REQUESTS | 655874

E2E Client Latency Observed:

P50 | 8ms
P90 | 12ms
P95 | 15ms

After installing the WAF:

THROUGHPUT | 1397 TPS
BYTES |<PHONE_NUMBER>
REQUESTS | 419445

E2E Client Latency Observed:

P50 | 12ms
P90 | 21ms
P95 | 23ms

Fairly substantial drop in throughput with this sample workload.
Overall I don't consider the performance degradation to be enough for me to scrap use of the application. But based on our testing there is certainly room to be desired. I believe I have minimized the ruleset and optimized as much as I can just about for an API Gateway's use case(feel free to correct me if you think up any irrelevant to my use case rules to drop that may improve these numbers/performance).
Anyways thanks for all the hard work put into this application and the widely adopted rulesets! I remember glancing at ModSec in 2016 for NGINX and decided against going down that rabbit hole at the time. Seems the application has matured and is much more flexible here in 2020 so cheers for that. Maybe a future release can dial in on areas for performance improvement (geared to help with throughput/latency bottlenecks associated with the WAF layer.)

Thank you for this extensive writeup. It's interesting to see what you can get out of it, when you drop rules that are of no interest to you.

Thank you for the notes @jeremyjpj0916. There is this wiki page - https://github.com/defanator/modsecurity-performance/wiki where @defanator carefully collects data on the performance of the engine with and without CRS loaded.


test(s2n-quic-core): revert to bigger length for Kani test

Resolved issues:
N/A
Description of changes:

In #2128, the length used in a Kani test was reduced from 16 to 4 because Kani was running out of memory (https://github.com/model-checking/kani/issues/3030). This issue is now resolved, so reverting to the previously used length of 16.
With Kani 0.54.0, memory usage for this harness does not exceed 1.5 GB.
Call-outs:

Testing:

N/A
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Thanks for following up! Happy to see this fixed :)


Icon Name not shown on hover

Current State :
In the current state of the figma plugin, when a user tries to hover to an icon, the icon does not shows name of the icon.
https://user-images.githubusercontent.com/67755381/159121866-d3d9f157-842f-4bc8-883f-11008ee717a2.mp4
Expected State :
When an icon is hovered, its name should be displayed on hover.

I am working on this issue.


Error: UPGRADE FAILED: create: failed to create: Secret "sh.helm.release.v1.prometheus.v9" is invalid: data: Too long: must have at most 1048576 bytes

Error: UPGRADE FAILED: create: failed to create: Secret "sh.helm.release.v1.prometheus.v9" is invalid: data: Too long: must have at most 1048576 bytes
was thrown on helm upgrade/install
The reason for this message had nothing to do with secret. It was a large xxxx.tgz file mistakendly placed in the working directory where the helm install was called. There were no dependency on values.yml and so on so they are not added to the issue.
Output of helm version:
helm version
version.BuildInfo{Version:"v3.5.3", GitCommit:"041ce5a2c17a58be0fcd5f5e16fb3e7e95fea622", GitTreeState:"dirty", GoVersion:"go1.15.8"}
Output of kubectl version:
kubectl version
Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:26:26Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:20:00Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.): DigitalOcean

The secret created by Helm is a record that helps Helm perform many tasks, such as rollback or to show information about a release. There are cases where that file could accidentally get added to the record Helm stores as data like that is sometimes stored because it's needed for other functions.
If it's something in there by accident, I would suggest deleting it. If it shouldn't be part of the delivered payload I would suggest adding it to the .helmignore file.

@Eljah isn't it possible to do the file size check and if they are larger than 1Mb to handle that like a outside accidental file, definitely not the true Helm secret? And throw the error about the wrong file with its name and location?

@Eljah That's a great question. The problem is that the file size is not consistent. For example, the limit has to do with etcd rather than Kubernetes. k3s uses sqlite and it's possible to configure etcd with a different size limit (there are tradeoffs do doing this). There is not an easy way for Helm to query the size to know the limit ahead of time.

@mattfarina ok, makes sense. But if there would be possible to have two reasons for the exception, the designed and the occasional one, would it be possible to add one more sentence to the error message and advice a user to check the directory for the accidentally placed tar files, not related to the helm secret?

I also spent a bit too much time to find that the error actually is "Helm stores the full chart as a Secret into K8s to allow rollback and other Helm features, but this chart is too big to fit within your K8s's specific Secret size limits". Please consider this an additional vote for a more explanatory error message.
There are already tickets to prevent this issue altogether, I'm just asking for the error message to be clearer to the end user.

I'm still interested in this getting improved.

Hi, me too

same here

I'm still interested in the error message UX getting improved.

I too would very much like to see a better warning. I'm pretty experienced with Helm (~3 years professional experience) but this error still throws me for a loop every now and then.


Error: More than one instance of bitcore-lib found

Uncaught Error: More than one instance of bitcore-lib found. Please make sure to require bitcore-lib and check that submodules do not also include their own bitcore-lib dependency.
at webpack:///~/bitcore-mnemonic/~/bitcore-lib/index.js:12:0 <- index.js:84076
Code:
bitcore.versionGuard = function(version) {
  if (version !== undefined) {
    var message = 'More than one instance of bitcore-lib found. ' +
      'Please make sure to require bitcore-lib and check that submodules do' +
      ' not also include their own bitcore-lib dependency.';
    throw new Error(message);
  }
};

See https://github.com/bitpay/bitcore/issues/1454  or  https://github.com/bitpay/bitcore/issues/1366

Hi @sc0Vu any progress ? I have the same issue here

@JellyBool yup, I just remove the lightwallet package and using yoethwallet.
Will merge into master soon!


Blog for week 1

@kmehant @Ammoniya @1209973 My blog for week 1 of the coding phase has been published.
Blog link: https://medium.com/scorelab/gsoc21-scan8-user-interface-2331eeea4272

LGTM to me!
Let's wait reviews from other mentors.


Compiler variable type bug

Hi, if you try to compile this code, you will get a nasty error (I tried to search for it, but I'm not sure whether it is related to the issues I have found):
class Foo
  @bar = @baz = 1

  def initialize
    puts "#@bar, #@baz"
  end
end

foo = `Foo.new

I have tested this code on my Ubuntu 16.04 desktop with Crystal 0.23.1 [e2a1389] (2017-07-13) LLVM 3.8.1 and on playground (link) they both have yielded the same result:
BUG: `` at  has no type
0x5bb9fe: ??? at ??
0x7a237d: ??? at ??
0xd49bae: ??? at ??
0xd48e8d: ??? at ??
0xd3f528: ??? at ??
0xd37027: ??? at ??
0xd1a432: ??? at ??
0xd2fc41: ??? at ??
0xd170b4: ??? at ??
0xd16b69: ??? at ??
0xd29913: ??? at ??
0xd2d99c: ??? at ??
0xd37044: ??? at ??
0xd1a432: ??? at ??
0xd2fc41: ??? at ??
0xd170b4: ??? at ??
0xd16b69: ??? at ??
0xcd7cdb: ??? at ??
0x102b0e5: ??? at ??
0x1098cbe: ??? at ??
0x611436: ??? at ??
0x5e1dac: main at ??
0x7f4dd7c1e830: __libc_start_main at ??
0x5bb20d: ??? at ??

Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues

Putting @bar = @baz = 1 into the constructor works.
Your puts string should probably be  "#{@bar}, #{@baz}"
class Foo
  def initialize
    @bar = @baz = 1
    puts "#{@bar}, #{@baz}"
  end
end

foo = Foo.new

carc.in

Duplicate of #3368


incorrect forge size check

When writing data to a lv2 atom port, forge.h does no correctly check available port-buffer-size, leading so to segfaults.
a potential fix is to change forge.h
@@ -298,6 +298,6 @@
 	} else {
 		out = (LV2_Atom_Forge_Ref)forge->buf + forge->offset;
 		uint8_t* mem = forge->buf + forge->offset;
-		if (forge->offset + size > forge->size) {
+		if (forge->offset + size >= forge->size) {
 			return 0;
 		}
 		forge->offset += size;

This was found by re-using an atom-port, writing to it twice.
if the first write is an empty sequence, the "capacity" is 8 bytes. Trying to write a new sequence segfauls in lv2_atom_forge_raw.
see also https://github.com/sadko4u/lsp-plugins/issues/28#issuecomment-462171971
example code to reproduce: https://gist.github.com/x42/b82769d2b3567b0a5c77c2bd857326ae

37e64e9

Those tests don't seem to catch overflows in lv2_atom_forge_raw(). e.g. when the buffer is too small to even write a sequence-head or frame_time.
This can happen e.g. in Ardour when a Plugin has an Atom output that is used for both MIDI and GUI. When the port is unused, Ardour provides a scratch-buffer and that may be overwritten.
2nd time around (LV2_Atom_Sequence*) port)->atom.size is 8 [bytes] (just the size of the seq-head),  yet to write that again fails. See the linked gist.

They do, it attempts to write a literal to a buffer with 1 byte.
I don't understand what you mean with this "second time around" stuff.  It sounds like the linked example snippet, where you are using the size of a written atom as the capacity to write to it again.  That's wrong, not a forge bug.  You will get crashes, because you are telling the forge there is more space available than there actually is.
Ardour is required to initialize any atom passed to a plugin as an output correctly, as a Chunk with size set to the correct capacity.  You can not do this again once the plugin has been called without resetting the buffer, and the plugin can not forge to it once then use the same init code again because the capacity is lost.
If you want to rewrite an atom output in a plugin for some reason (you shouldn't), you need to store the original capacity somewhere.  Using the initial size as a capacity like this is a bit janky, but it seemed better than a different wrapper type in the buffer since you can point plugins directly at any atom, as with audio.

Ah, I see now.  I was confused by the title and suggested fix.  This was actually an issue with pushing containers like sequences to the forge, then continuing to write even if it overflowed.  The frame in the forge pointed to NULL which would result in an eventual crash somewhere (further writing or popping).
c0a3fc6 fixes this and adds tests that I think cover all these cases.

Thanks!


SegmentedControl - Updates

Hi, I was wondering if you had any updates or timelines on the SegmentedControl. There are some great resources where people have submitted SegmentedControl Views using a radio box, but it's still missing stuff.

Hi @Geramy
There is a segmented control example in the template that is built with SelectionView. I haven't implemented a material design appearance yet but it's working functionally.

Here is the usage in UraniumUI:
https://github.com/enisn/UraniumUI/blob/e212bf58e98af248590629d474591683d18faa0b/templates/default/MyCompany.MyProject/MainPage.xaml#L153-L158


Velocity-level constraints / jacobians

Move from spring-like impulses for joint constraints to velocity level updates by calculating jacobians of the equality/inequality constraints.

We tried this and it did not work as well as PBD, so moving forward we'll be focusing on that approach


MAINT: Fix docs

Should fix https://app.circleci.com/pipelines/github/mne-tools/mne-python/17037/workflows/666aa313-3632-4165-b6d0-f5fc4deab93b/jobs/50533

This fix is trivial and CircleCI is happy, so in it goes


Consider adding a script to generate test_all.dart files

See https://github.com/dart-lang/build/blob/master/tool/create_test_all.dart as an example.
Since its required that you only run a single dart file, many packages today maintain a file which imports all their tests and runs them. This can easily get out of sync, and is usually pure boilerplate. A simple script like the one I linked to above can generate it automatically for most use cases.

Isn't this a test package issue?
When test and pub are changed so pub run runs in an isolate instead of a new process (which I is planned AFAIK) then pub run test should allow to collect coverage information and pub run test would take care of running all test files.

I do see https://github.com/dart-lang/test/issues/36 in the test package. The solution I linked above works today though (for vm tests at least). I am fine with just copying that into any of my packages that need it in the meantime though.

I think that approach would lead to the issue that a file is being generated and reports are generated for that file and sent to coveralls. On coveralls then there would be a file that is not tracked in git which would lead to confusion in my opinion. A more clean approach is to really have one file that is also tracked in git that fulfills this purpose.


Investigate Minecraft's inbuilt profiling, add our own bits

Shift-F3 shows realtime profiling.
And/or: add profiling information to the console log. Optionally: return performance summary in the MissionEnd description.

Started using this in 6837bec26a726815bced801f9e413ac1f8ad1d9d to help investigate #263


taxonomy/update: Updating reference broken

e.g. https://de.serlo.org/entity/taxonomy/update/64855
When marking a checkbox and saving the form you get redirected back to the entity, but the update failed. Probably a CSRF token problem.

update: Not a CSRF problem, as no CSRF is needed in this form. Works fine with my account.
Perhaps its an authorization problem

update: Not a CSRF problem, as no CSRF is needed in this form. Works fine with my account.
Perhaps its an authorization problem.

Can't be reproduced by original author, so I'll close this for now...


brain is in ms when stc is in seconds

I'm trying to save a movie of an stc in with tmin etc and times in seconds but when I create a brain object it is converted to ms in ...data_dict['lh']['time']. Is this a bug or is there some conversion I need to do?

Please provide the exact code you are using. It is hard to say what you need to do if all the information we have is "with tmin etc".

Sure thing, let me know if you need to know more.
   group_avg_stc = sum([i[0] for i in subj_stcs]) / len([i[0] for i in subj_stcs])
    brain = group_avg_stc.plot(subject='fsaverage',hemi='split',
                  subjects_dir=fs_dir,views=['lat', 'med'],
                  config_opts=dict(width=2500, height=1400),
          colormap='jet')

group_avg_stc.tmin and tstep are in seconds. group_avg_stc.times is in seconds.
brain.data_dict['lh']['time'] is in ms.

This inconsistency was recognized, and was solved by introducing the time_unit parameter in mne (see mne docs) so I think this issue can be closed.


[BUG] Only Small Subset of Messages Are Appearing in Archive Viewer When Using DF Chat Enhancement

Module Version: v0.4.1
Describe the bug
When I run an archive on a set of messages, those messages are properly saved out to the JSON file with all of the functionality and meta data they should have. However, when I try to open those archives in the module, I'm not seeing all of the messages. I only see around... maybe 100? It's hard to count for sure--but I know many of my 700~ messages are not being shown.
To Reproduce
Steps to reproduce the behavior:

Download the attached log and move it to a location Foundry can access
Import the log using Configure Settings > Vauxs' Archives > Open Chat Archives > Add an existing archive
Open the archive, called "Test" (or whatever you imported the archive as)
Note that only a small subset of the messages appear, and scrolling to the top does not

Expected behavior
When you open up an archive and scroll up, it should continue loading more messages from the archive file until it reaches the top of the file
Browser:

Firefox v121.0.1
Chrome v123.0.6312.58

Foundry Version:
v11.315
Game System:
D&D 5E 3.03
Additional context
I don't get any errors in the console.
I imagine that it's doing an initial load, but further lazy loading isn't occurring properly for some reason.
P.S. Thank you for the work on this module! DF's chat archiving never worked for me at all, so this is already a huge improvement.
iWq2N4H3MOMKsmN3.json

Check for any other modules interfering with the module. I cannot replicate this issue.

Ah, I should have done a quick check without any first.
It seems like the problem is arising from some sort of conflict with DF Chat Enhancements (or maybe Chat Commander, its dependency).
I can probably get away without it since Midi QoL can duplicate one of its main benefits (chat coloring), but it might be something to investigate. I also know that DF Chat Enhancements isn't maintained actively anymore, so maybe the compatibility isn't a high priority.

It's 100% DF Chat Enhancements, as it has a now-very-outdated feature of limiting the rendering of the chat to 100 messages (Foundry v11 implemented that already).
Get rid of it! You have Karakara's Chat Enhancements for all other features, including chat coloring. https://foundryvtt.com/packages/karakara-chat-enhancements


The lack of ability to change the default service error handling

While investigating #573, I've found that ArmeriaServerBuilder#serviceErrorHandler is unused resulting in the lack of ability to change the default service error handler org.http4s.server.package$#DefaultServiceErrorHandler.

It was my mistake. 😅 ArmeriaHttp4sHandler should be fixed to use ArmeriaServerBuilder#serviceErrorHandler.

Yeah, the fix is quite straightforward. I'll drop a PR this weekend. But your review is highly appreciated! :)


Read data/images from directories when compiled to desktop

If there was an extension or addon similar to the Files extension that could allow data like .json to be read from anywhere INSIDE the project folder, it could allow for clearer organization.
Things that could be made easier if stored inside a directory instead of a global list inside the project:

Level data (good for tile based platformers with lots of levels)
Keybinds and other settings
Achievments or unlockables
Dialogue or other NPC/cutscene data
Decreased loading time now that less lists are loaded inside the project

Here's an example of what it could look like, used in that one game about the guy with blue hair

I don't know if this could create any security issues, maybe just restrict reading and writing to inside the project folder

You can just use the fetch extension. If you want a file directly from the same directory, just put the file name + extension (level.json for example) in the fetch extension block. Same with folders. Just do foldername/level.json .

Here is a explanation for how it works.
https://www.w3schools.com/jsref/api_fetch.asp

You can just use the fetch extension. If you want a file directly from the same directory, just put the file name + extension (level.json for example) in the fetch extension block. Same with folders. Just do foldername/level.json .

That makes more sense. I originally assumed that the fetch extension was ONLY for online content but I guess it can be used for local files as well. Thanks!

By the way, the fact that you used Friday Night Funkin as an example makes me feel like you want to make something like it with TurboWarp. I already did something like this if you need a starting point :)
I stopped working on it since I lost interest.
https://github.com/TheShovel/Funkin-on-the-web

Well, It was the only example I could pull up quickly. What I meant was something more like this:

(My save file for Pizza Tower.)

Ah, alright.
You can close the issue now, by the way!

One last thing, TW can read .ini files effectively, right? If I implement saves into a game I want to see if i can make one just like this.

Of course! It's just text after all. You can fetch anything.

One last thing, TW can read .ini files effectively, right? If I implement saves into a game I want to see if i can make one just like this.

There is no INI parser extension AFAIK, but you could always just implement one yourself.

Of course! It's just text after all. You can fetch anything.

can it also fetch audio/image files?

Of course! It's just text after all. You can fetch anything.

can it also fetch audio/image files?

yes, any file

You can just use the fetch extension. If you want a file directly from the same directory, just put the file name + extension (level.json for example) in the fetch extension block. Same with folders. Just do foldername/level.json .

Sorry for the bump, but I didn't want to create another issue on this since there is already so many of them open as of right now.
Does this still work for you? I just tried it with packaged versions of a test project and it didn't do anything whatsoever, no matter the type of package, I tried Electron Win32+64, Win64 and plain HTML with a sample text file in a folder called "mods", then tried it with the text file being moved into the root folder, still nothing...


⚠ Jenkins Plugins Artifactory has degraded performance

In 5ba244e, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1276 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 28d65a5 after 39 minutes.


lightning-loop: init at version 0.12.2

Motivation for this change
This adds the Lightning Loop Client to nixpkgs
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Cool, sorry for that.


chore: use structopt wrapper instead of building clap by hand

Type safe wrapper around clap. Supports everything clap does. Has exactly the same command output as it did before

wow structopt is a lot neater :heart_eyes:
Thank you!


Save the loop progress state by default

What does this PR do?
Save the Loop's progress tracking state in the checkpoint.
We don't load it yet as that will be a breaking change. To be done in a follow-up.
Does your PR introduce any breaking changes? If yes, please list them.
None
Before submitting

[n/a] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[n/a] Did you make sure to update the documentation with your changes? (if necessary)
[n/a] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you list all the breaking changes introduced by this pull request?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @borda @awaelchli @ananthsub @ninginthecloud @carmocca @justusschock

n00b question, where does this happen?

We don't load it yet as that will be a breaking change. To be done in a follow-up.

restore_loops seems to just proceed as long as loops entry in ckpt dict is present?


Exporter exports nothing unless you symbolize keys

Had the same issue, just worked through it by using deep_symbolized_keys


KEYs

Where can i get the training and prediction key

Visit https://customvision.ai and sign up for an account.
Create a project (you can delete it afterwards) and open it.
Click on the 'gear' icon at the top right hand side, which will list your keys.
Note: keys are per-user, not per-project, but currently the only way to see them on the website is in the context of an existing project.


CriticalFinder

[ ] Convert critical issues from inspections into computationally defined variables
[ ] Identify input datasets
[ ] Prioritise cost/benefit ratios
[ ] Find critical issues nationwide

[ ] Conflict with motor traffic at side roads / priority junctions
[ ] Conflict with motor traffic at signal controlled junctions and roundabouts
[ ] Collision alongside or from behind
[ ] Trip hazard
[ ] Conflict with kerbside activity (parking, loading, risk of 'dooring' and bus stops)
[ ] Risk of crossing conflicts
[ ] Standard of crossing facility
[ ] Speed of traffic (where cyclists are not separated or pedestrians crossing uncontrolled)
[ ] Total volume of traffic (where cyclists are not separated or pedestrians cross uncontrolled)
[ ] Required crossing speed (risk of pedestrians coming into conflict with traffic)
[ ] Clear walking spaces free of obstructions and furniture, reducing risk of pedestrians walking in the carriageway.

[ ] Publish results

This issue builds on #6 and will add values to the database on infrastructure we hold.

Four critical issues missing:

Effective width next to tram line on a straight run
Crossing angle (between cyclist desire line and tram tracks)
Cycle surface defects: non cycle friendly ironworks, raised/sunken covers/gullies
Pedestrian surface defects: non flush tables, misleading tactile information, cracked pacing, slip-risks present from covers, steep slopes

Crit   Issue #
Critical Issue
Process / Data   Required
TH Comment
Ease of   Computation (0 hard, 10 very easy)
TH Comment

6
Risk   of crossing conflicts
1)   Is the traffic flow > 8000vpd?     2) Are there formal crossings present?     3) What is the largest gap between formal crossings? If over 400m then   critical.
There   is also another critical scenario for roads with <8000vpd which will be   very hard to compute ("Are any obvious pedestrian desire lines blocked   by parked cars?")          For formal crossings definition (see attached Crossings diagram)
8
If   we can identify the relevant crossing types and get flow data, this might be   fairly straightforward.

11
Clear   walking spaces free of obstructions and furniture, reducing risk of   pedestrians walking in the carriageway.
1)   Is the clear (unobsctructed) footway width <1.0m?     2) Is the clear (unobstructed) footway width <1.5m for over 6m?     3) If yes to either of those, then critical
Absolute   minimum acceptable is 1m but only for a length of 6m. Desire minimum is 1.5m
8

7
Standard   of crossing facility
1)   Is the traffic flow > 8000vpd?     2) Are there two lanes of traffic to cross?     3) What form of crossing is provided? If uncontrolled then critical.          A) Identify signal junctions     B) Are there any arms with no green man for pedestrians? If yes then   critical
Two   separate checks: For links it's 1,2,3; for junctions it's A,B.          There is also a 'gaps in traffic' element to this but for ease I would   ignore that (with Brian's permission)          For uncontrolled crossings definition (see attached Crossings diagram)
7
The   link check is more straightforward than the junction check

8
Speed   of traffic (where cyclists are not separated or pedestrians crossing   uncontrolled)
1)   Are cyclists mixed with traffic? AND/OR Do pedestrians cross   uncontrolled?     2) If yes, is the 85th percentile speed over 60km/h?

7
Is   85th percentile speed data available?          Identifying whether peds cross uncontrolled might be challenging

9
Total   volume of traffic (where cyclists are not separated or pedestrians cross   uncontrolled)
1)   Are cyclists mixed with traffic? AND/OR Do pedestrians cross   uncontrolled?     2) If yes, is the total volume of traffic > 1000vpd?

7
Identifying   whether peds cross uncontrolled might be challenging

13
Crossing   angle (between cyclist desire line and tram tracks)
1)   Is there a tramline?     2) Are cyclists expected to cross the tramline at an angle of less than 60   degrees?

7

3
Collision   alongside or from behind
1)   Are cyclists mixed with traffic in nearside lanes?     2) If yes is the lane width between 3.25m and 3.9m?

6
Depends   on accuracy of satellite / other data for lane widths?

5
Conflict   with kerbside activity (parking, loading, risk of 'dooring' and bus stops)
1)   Is there a cycle facility next to parking/loading?     2) Is there a 0.5m buffer between them?

6
Depends   on accuracy of satellite / other data for lane widths?

12
Effective   width next to tram line on a straight run
1)   Is there a tramline?     2) Is there <2.4m between the kerb and the tram rail and are cyclists   expected to cycle in that space? If yes then critical

6

1
Conflict   with motor traffic at side roads / priority junctions
1)   Identify side roads / priority junctions     2) Are cyclist and ped movements protected?     3) If not, what are the turning counts (motor vehicles per day)
Protected   = zebra crossing, continuous footway, modal filter, priority maintained for   cyclists          (2) is basically "are there red movements if a JAT check were   performed?"
5
I   imagine (1) is easy, (2) is hard, and (3) depends on what flow data is   available.

2
Conflict   with motor traffic at signal controlled junctions and roundabouts
1)   Identify signal control junctions / roundabouts     2) Are cyclist and ped movements protected/separated in time/space?     3) Turning counts (motor vehicles per day) clashing with any unprotected   ped/cycle movements
(2)   is basically "are there red movements if a JAT check were   performed?"
3
Same   comments as crit issue #1 but these junctions are more complex so (2) is   hard.

10
Required   crossing speed (risk of pedestrians coming into conflict with traffic)
1)   Locate signalled pedestrian crossing     2) How long is the crossing?     3) How much time is there for the stage?     4) What speed to peds have to move to get across in time? If faster than   1.2m/s then critical.
This   is actually quite a rare critical as most crossings are designed for 1.2m/s   or slower.
3
Finding   stage times might be hard, everything else can probably be approximated   through satelite imagery?

4
Trip   hazard
1)   Are the level differences of greater than 20mm in the walking   environment?     2) Is there any colour contrast to help identify them?

2
Robin   mentioned LIDAR but can see this one being challenging!

14
Cycle   surface defects: non cycle friendly ironworks, raised/sunken covers/gullies
1)   Are there any major defects, such as those listed, in the cycle space?

1

15
Pedestrian   surface defects: non flush tables, misleading tactile information, cracked   pacing, slip-risks present from covers, steep slopes
1)   Are there any major defects, such as those listed, in the pedestrian space?

1

Many thanks @tom-ate-holcroft v. useful!

Wow, this is a goldmine! What's the overall goal for us here? To make a guided flow in something ATIP-like to report / record these problems, to try and auto-discover some of them, something else?

Goal: TBC. Long term: to port full functionality of Excel-based tools into a web app architecture. Short term: simply find the critical issues and generating outputs that show where they are could be key.


Issue: Can't delete echomail

When posting a text in a fidonet conference, I can't delete my own text before it's been tossed and sent away to the hub.
The error message is very confusing also.   Example:
Text 945  Möte: General Discussion (Fsxnet)    16 Mar 16  18:24:51
Skriven av Joacim Melin (21:1/116.0)
Till: Paul Hayton
Ärende: Re: AMIBinkd
!bo
--- NiKom v2.0.0

Origin: Delta City (deltacity.se, +46-8-6058800) (21:1/116.0)

(Slut på text 945 av Joacim Melin)
(Se) tiden --> rad te 945
Du kan bara radera texter av dig själv!
(Se) tiden -->

Deleting texts is not supported for FidoNet forums. There are several problems with being able to delete a Fido text.

I'm not sure whether NiKom can tell whether a text has been tossed and sent out or not. If it has it shouldn't really be deleted.
If the text is not the highest text there will be a gap, which is not supported.
If the text is the highest and gets deleted then the next text (written or imported) will get the same number. If someone had already read the old deleted text they will never get to read the new text with the same number.


Duplicate Members after editing them in AdminCP

With application* disabled:
Search member in AdminCP > Edit member > Save member > Search same member > Shows 1
With application* enabled:
Search member in AdminCP > Edit member > Save member > Search same member > Shows 2 (duplicate created)
No idea why, but for some reason it creates duplicates...
When either of the duplicate members is deleted (and content they created is kept), there doesn't seem to be any changes to the member or the created content.
Also, if any of the duplicated members are edited and saved again, it creates another duplicate (an infinite amount, if you wouldn't manually delete duplicates after editing those members)
* = Discord Integration 1.2.0 Alpha 3 & Register Module

This should be fixed in 4.2.2 (I believe it was a known bug) @deknarf

@deknarf Is this still happening with the latest IPS version?

Latest version was working for me. Though the one before it had this bug. Will try a clean install and report back.

still happens for me, v4.2.5


[TRG 4.05 - Container Registries] Publish Docker image on hub.docker.com

Adjust your workflows for docker image build to publish image on Docker Hub, please refer to TRG 4.05 - Container Registries.
https://github.com/eclipse-tractusx/managed-identity-wallet/blob/93f9ade777fde3a081636790b8152aa32b50dc5b/.github/workflows/service-build.yaml#L8-L10
Please ensure to tag your docker image to match requirements from TRG 4.01 - Image Tagging.

After this is done enable chart lint testing again in chart-lint.yml

I can currently not find any docker image build + publish workflow. I also cannot find a released image on DockerHub yet. This is blocking #53


Add a test to confirm we are not vulnerable to CVE-2021-23337

See #2911.

Coverage remained the same at 95.197% when pulling a4cc7c065517f0fa132217161794b015ca537a5d on jgonggrijp:master into 745e9b7314064e66a7257f9b361030e6055980b8 on jashkenas:master.


Set up circle, spin down travis

They say travis is dead. More to the point, it doesn't have python 3.7 support.

I have the main builds going on GH actions now so I'm comfortable closing this issue. However, I don't have a replacement for ppc tests and the Travis tests are actually broken, so I'll have to make a follow-up issue.


Compile error when trying to send a message

Describe the bug
Xcode can no longer compile the swift code, ones room.sendTextMessage is called.
To Reproduce
Steps to reproduce the behavior:

Create a MXSession like in the examples
select a room from the MXSession.rooms value
call sendTextMessage on the room
Xcode cannot compile anymore (Trace/BPT trap: 5)

(things like sendTypingNotification on the room works)
Expected behavior
should compile and send a text message
Desktop (please complete the following information):

OS: macOS 11.0.1 (apple silicon M1)
Version [0.17.4] (swift)

It works while using __sendTextMessage, not sure why that works.
room.__sendTextMessage("foobar", formattedText: nil, localEcho: nil, success: {p in print(p)}, failure: {_ in})