Django 1.9 deprecation warnings

Hi,
I'm working on a project that uses django-flatblocks and we recently updated to Django 1.8. I'm trying to get rid of all RemovedInDjango19Warning messages, and put together a pull request to silence one in flatblocks aswell: https://github.com/funkybob/django-flatblocks/pull/14
If you want me to change something in it, just give me a shout!

Patch looks good... any other warnings, please open new tickets or a PR :)


LLM training fails with MPI

when running pretrain.py with 1 or 4 GPUs and the DDPStrategy as described in the docs, I get the following error
"PyTorch/1.12.0-foss-2022a-CUDA-11.7.0/lib/python3.10/.../torch/distributed/utils.py", line 113, in _sync_module_states _sync_params_and_buffers(
"PyTorch/1.12.0-foss-2022a-CUDA-11.7.0/lib/python3.10/.../torch/distributed/utils.py", line 131 in _sync_params_and_buffers
dist._broadcast_coalesced(
IndexError: map::at
srun: error: tasks 0, 2-3: exited with exit code 1

This is related to MPI: in pretrain.py, we're using the following strategy:
ddp = DDPStrategy(process_group_backend="mpi")

We use the following bash scripts:
https://github.com/odissei-lifecourse/life-sequencing-dutch/blob/main/code/llm/slurm_scripts/pretrain_multigpu.sh
https://github.com/odissei-lifecourse/life-sequencing-dutch/blob/main/code/llm/slurm_scripts/pretrain.sh

this definitely seems to be an issue with MPI. when we run with 1 GPU and use the default strategy (ie, do not specify to use strategy=DDPStrategy(process_group_backend="mpi"), the code works fine.

Question from SURF: why do we not use the 2023 software stack? It's compiled against a more recent version of MPI and should be better supported.
We can try to move to the 2023 stack. @dakota0064 , @tanzir5 , do you any reason right now why this might not be possible?

Hi @benczaja , I don't have any publicly available data at the moment, but am working on it. I'll let you know when they are ready 1.5-2 weeks from now). see #40

Hi @benczaja , I have the data ready now on snellius, but currently a dependency problem: #32 . I'll keep you posted.

@benczaja , the data are now ready, and the code on the branch llm-pipeline-snellius works as follows.
I want to merge the branch soon, but it requires some tidying. In the meantime, feel free to work with the code on the branch. The following should work
git clone git@github.com:odissei-lifecourse/life-sequencing-dutch.git
cd life-sequencing-dutch
module purge
module load 2022
module load Python/3.10.4-GCCcore-11.3.0
module load SciPy-bundle/2022.05-foss-2022a
module load h5py/3.7.0-foss-2022a
module load matplotlib/3.5.2-foss-2022a
python -m venv .venv
source .venv/bin/activate
pip install -r requirements/snellius.txt
pip install performer-pytorch==1.1.4

and then you can run the code either with
cd life-sequencing-dutch/src/llm
sbatch slurm_scripts/pretrain.sh

or within an interactive job
cd life-sequencing-dutch/src/llm
source slurm_scripts/pretrain.sh
python -m src.new_code.pretrain.sh projects/dutch_real/pretrain_cfg.json

In src/llm/new_code/pretrain.py you see some constants at the top of the script that you can adjust for how training should be done. I've not had time to try all options out, so there may be bugs.
I tried to use MPI, but I got RuntimeError: Distributed package doesn't have MPI built in. MPI is only included if you build PyTorch from source on a host that has MPI installed., the reason for which is I think that we use pytorch from pip and not from modules.
Let me know if you have questions.

Hey we need to either build PyTorch from source or use our system Pytorch (which was built from source). Because pip will not install PyTorch with MPI functionality.... Or at least I don't know how to instruct pip to do so.
https://discuss.pytorch.org/t/distributed-package-doesnt-have-mpi-built-in/196094

OK it looks as though certain communication routines are not implemented in the DDP strategy MPI

https://pytorch.org/docs/2.4/distributed.html#backends (see the question marks)
The dist._broadcast_coalesced error reported above I think falls into this. So its how PyTorch lightning has implemented the collective communication calls of PyTorch. MPI is just lacking (my guess). They seem to give more support to NCCL and gloo https://pytorch.org/docs/2.4/distributed.html#which-backend-to-use . They simply mention MPI no-where.

So the solution for now is just use gloo as the ddp strategy.

I have re-written pretrain.py a bit to easily take in arguments https://github.com/odissei-lifecourse/life-sequencing-dutch/blob/4d10b2da9989959d8adbcfbc103773718dc6b36a/src/llm/src/new_code/pretrain.py#L144

It can now be used like this https://github.com/odissei-lifecourse/life-sequencing-dutch/blob/4d10b2da9989959d8adbcfbc103773718dc6b36a/src/llm/slurm_scripts/pretrain_multigpu_singlenode.sh#L23

Have done some simple bench-marking seeing the difference between gloo vs NCCL. NCCL reports ~ 3.51 iterations/s and gloo 3.3 iterations/s this is not a glaring difference, and somehting I think we can live with in the OSSC

The biggest performance increase would just be using more of the GPUs. The GPUs are simply under- utilized. I see about the usage of about 30% of the GPUs when running. But I assume that this is just model and data dependant. So when the model size is increased and/or the data is increased then we will really use the GPUs. THEN NCCL will probably be more of an issue.

GPU utilization for context

I reproduced this for ddpstrategy in (gloo, auto, ddp). I get 1.65 it/s with gloo and 1.75 with the other two, and around 30% GPU usage.

I see about the usage of about 30% of the GPUs when running. But I assume that this is just model and data dependant. So when the model size is increased and/or the data is increased then we will really use the GPUs. THEN NCCL will probably be more of an issue.

What I'll do next is increase the utilization of the GPUs used. This means bigger batch size, more parameters and/or higher precision. Increasing the size of individual samples would be an option too, but that's not straightforward to do at the moment.

I realize we forgot to use srun. So, my plan is now

https://lightning.ai/docs/pytorch/stable/clouds/cluster_advanced.html -- use srun! then compare differences again
make a table comparing

model size/batch size
backend used

then, check also

GPU utilization (tensor cores)
do we get a speedup with multiple GPUs?

depending on that, discuss with Tanzir on the next steps

Here are some comparisons from using srun python -m ...

auto
auto
gloo
gloo

model size
N gpus
speed
gpu utilization
speed
gpu utilization

small
4
2.94 it/s
15.00%
2.97 it/s
10.00%

large
4
2.00it/s
30-50%
1.92 it/s
30-50%

large
2
2.01 it/s
40.00%
1.95 it/s
30.00%

(note, using srun makes the code faster, but not differentially so between nccl and gloo backends)

the gpu utilization is always fluctuating a log (see Ben's figure above), which suggests that either there is a bottleneck in data loading, or there are other inefficiencies. for data loading bottleneck, I need to check if we're CPU-constrained: https://github.com/Lightning-AI/pytorch-lightning/issues/18650. I have quickly tried what is discussed there but found no difference.
For other inefficiencies, my colleagues suggested to profile the code with Nsight
mixed precision/tensor core utilization still not investigated.
the lack of speedup between 2 and 4 gpus is puzzling

The conclusion from here is to not use MPI. I'm closing this in favor of #74 .


Error 403 when using [JWPlatform], Tokenized JSON,Possible Solution.

Checklist

[x] I'm reporting a site feature request
[x] I've verified that I'm running youtube-dl version 2<IP_ADDRESS>
[x] I've searched the bugtracker for similar site feature requests including closed ones

Description

I've been trying to download some files of off my institute's website. They use jwplayer with the {media_id}-{player_id}.js format. The .js file contains the exact URL (token included) to access the required JSON metadata but youtube-dl doesn't work even if I have provided the actual tokenized version of the JSON link.
Caused when using the webpage as input.
youtube-dl -F https://cdn.jwplayer.com/players/2YD5fq8v-dVg6Al1Y.html -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://cdn.jwplayer.com/players/2YD5fq8v-dVg6Al1Y.html', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg git-2020-04-17-889ad93, ffprobe git-2020-04-17-889ad93
[debug] Proxy map: {}
[JWPlatform] 2YD5fq8v: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Caused by using the tokenized link to JSON.
youtube-dl -F https://content.jwplatform.com/v2/media/tcg4Rnxm?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZWNvbW1lbmRhdGlvbnNfcGxheWxpc3RfaWQiOiJvM3haWTgxQiIsInJlc291cmNlIjoiL3YyL21lZGlhL3RjZzRSbnhtIiwiZXhwIjoxNTk0OTc5NTIwfQ.zQzAXq9KLDueU01YlX5uZ891PO-L1yDn5Teuy90Y7ko -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://content.jwplatform.com/v2/media/tcg4Rnxm?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZWNvbW1lbmRhdGlvbnNfcGxheWxpc3RfaWQiOiJvM3haWTgxQiIsInJlc291cmNlIjoiL3YyL21lZGlhL3RjZzRSbnhtIiwiZXhwIjoxNTk0OTc5NTIwfQ.zQzAXq9KLDueU01YlX5uZ891PO-L1yDn5Teuy90Y7ko', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg git-2020-04-17-889ad93, ffprobe git-2020-04-17-889ad93
[debug] Proxy map: {}
[JWPlatform] tcg4Rnxm: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

A solution I've come up with works on all video preview page that is obtained by using any link of the format https://cdn.jwplayer.com/players/{media_id}-{player_id}.html
A js parser(noob's implimentation below) that extracts the var jwConfig = {} from the script_tag and subsequently extracts the correct media_uri/playlist_uri would do the trick.
The .js data(request.content) should be treated as a str/bytes object and split using the (";") delimiter. The element [-5] (please verify) should be the appropriate variable (jwConfig). Removing the var jwConfig =  (incl. spaces) then using json.loads()  to get a dictionary. This dictionary has a playlist key which contains the JSON metadata. (might also have to replace the // with https://)
Probable solution for the preview page of most media files (jwplayer-hosted/cloud-hosted)
import json
import requests
#
main_url = "https://cdn.jwplayer.com/players/<media_id>-<player_id>.html"

#
def get_playlist(js_text): # besure to change split wrt the r.content from requests.
    _list = js_content.split(";")
    jwConfig = _list[-5]
    _dict = jwConfig.split("= ", 1)[-1]
    _config = json.loads(_dict)
    _playlist = _config['playlist']
    playlist_url = _playlist.replace("//", "https://")
    return(playlist_url)

# this is the input preview webpage. (and only the preview webpage) (only use if 403?)
resp = requests.get(main_url)

def get_js_from_webpage(response):
    resp_str = response.decode('ascii', "ignore")
    index = resp_str.find("script.src = ")
    return(resp_str[index + 14 : index + 73]) # i have no idea how to do this part
    #(are hardcoded values ok ?) media id and player id have constant length.

url = get_js_from_webpage(resp.content)
js_content = requests.get(url).text
playlist = get_playlist(js_content)
print(playlist) # all of this can be done under one function i guess.
# works fine but im a noob so
# edited to add
playlist_json = requests.get(playlist).json()
mpeg_url = playlist_json['playlist'][0]['sources'][0]['file']
print(mpeg_url)
# this gives me proper download. in this case links are valid for 3 hours

When visiting the url returned after this, we get a JSON document similar to the ones we get when using a untokenized media.
{
   "title":"Hb Xi C Ch-8 Chem Bond Cl 18",
   "description":"",
   "kind":"Single Item",
   "playlist":[
      {
         "title":"Hb Xi C Ch-8 Chem Bond Cl 18",
         "mediaid":"tcg4Rnxm",
         "link":"https://content.jwplatform.com/previews/tcg4Rnxm?exp=1594979760&sig=ce7935627dda7be1085357c080e22c3b",
         "image":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=720",
         "images":[
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=320",
               "width":320,
               "type":"image/jpeg"
            },
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=480",
               "width":480,
               "type":"image/jpeg"
            },
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=640",
               "width":640,
               "type":"image/jpeg"
            },
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=720",
               "width":720,
               "type":"image/jpeg"
            }
         ],
         "duration":3562,
         "pubdate":1588320989,
         "description":"",
         "recommendations":"https://content.jwplatform.com/v2/playlists/o3xZY81B?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZWxhdGVkX21lZGlhX2lkIjoidGNnNFJueG0iLCJleHAiOjE1OTQ5Nzk3NjAsInJlc291cmNlIjoiL3YyL3BsYXlsaXN0cy9vM3haWTgxQiJ9.phjoZ4M9PYpe8d3nWeO7v3ZpDFLlYbGA7TmEJx-CaxM",
         "sources":[
            {
               "file":"https://content.jwplatform.com/manifests/tcg4Rnxm.m3u8?exp=1594979760&sig=b3fb6ae11e470fd582017ff9382a438c",
               "type":"application/vnd.apple.mpegurl"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-LKUbW2zZ.mp4?exp=1594979760&sig=f4ebbde57bf9f3fe02a5a7e42c9ab88c",
               "type":"video/mp4",
               "height":180,
               "width":320,
               "label":"180p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-6SUujk2h.mp4?exp=1594979760&sig=c88111d374dd9a0fafa5739d09e76ff0",
               "type":"video/mp4",
               "height":270,
               "width":480,
               "label":"270p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-RfJ1YLUn.mp4?exp=1594979760&sig=2a940230ec914d6524dfeb7297f28f37",
               "type":"video/mp4",
               "height":720,
               "width":1280,
               "label":"720p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-eFTQgXDM.m4a?exp=1594979760&sig=457512da11ab968945b9512a82ca8b32",
               "type":"audio/mp4",
               "label":"AAC Audio"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-fD4bxXyo.mp4?exp=1594979760&sig=1b1335fb87eafaacc2e752e04cdd2ebb",
               "type":"video/mp4",
               "height":1080,
               "width":1920,
               "label":"1080p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-VBGx6SI9.mp4?exp=1594979760&sig=55df053c9fe7234755db817553fbe20f",
               "type":"video/mp4",
               "height":360,
               "width":640,
               "label":"360p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-PFZiwg3E.mp4?exp=1594979760&sig=dbe74ccf154696875f2c6d64db731ac6",
               "type":"video/mp4",
               "height":540,
               "width":960,
               "label":"540p"
            }
         ],
         "tracks":[
            {
               "file":"https://content.jwplatform.com/strips/tcg4Rnxm-120.vtt",
               "kind":"thumbnails"
            }
         ],
         "variations":{

         }
      }
   ],
   "feed_instance_id":"11aeef4f-cf4b-4606-8494-01c8959441be"
}

A snippet of the js (the important bit) is included below.
var jwConfig = {
  "aboutlink": "<redacted>",
  "abouttext": "<redacted>",
  "aspectratio": "16:9",
  "autoPause": {},
  "autostart": false,
  "cast": {
    "appid": "00000000"
  },
  "controls": true,
  "displaydescription": true,
  "displaytitle": false,
  "flashplayer": "//ssl.p.jwpcdn.com/player/v/8.16.3/jwplayer.flash.swf",
  "height": 360,
  "key": "<redacted>",
  "logo": {
    "file": "https://assets-jpcust.jwpsrv.com/watermarks/Qnqjvg74.png",
    "hide": true,
    "margin": "10",
    "position": "top-right"
  },
  "mute": false,
  "ph": 3,
  "pid": "dVg6Al1Y",
  "playbackRateControls": true,
  "playlist": "//content.jwplatform.com/v2/media/2YD5fq8v?token=<redacted>",
  "preload": "none",
  "repeat": false,
  "stagevideo": false,
  "stretching": "uniform",
  "width": "100%"
};

hi bro i am also doing same now ? do have any gdrive link of all of them ?

@priyanshurohilla2004 Please contact me on a different channel, This issue is not the place to discuss this.

ya i have sent you mail as well as msg on reddit and twwitter


The red point isn't following the mouse after open zoom window when using the annotation tool

Hi, I use the annotation tool to annotate images, and find when the zoom window is open, the red isn't following the mouse, as shown in image below. I can't figure out the reason. Is this normal? Could you help to fix it? @mcordts

Actually that's on purpose to allow for a precise annotation.
If you don't like this behavior, you could try to set the sensitivity to 1 in line 2042

Thank you. I got it. That really does help a lot.

@mcordts the red point is following the curson on setting the sensitivity to 1 but the zoomed image keeps on moving along with the cursor which doesn't allow precise annotation. Is there some way by which I can stop the movement of the window.


[feat] Implement to register webhook automatically

Register webhook when application is created
Add git config to application spec
Get webhook server address simply from configmap (will modify later)

/approve


cuda-version

Hi!:
I'd like to know which cuda your great programer is testing on.

The training code was tested on CUDA 10.0. Testing has been run on CPU and CUDA 9.2 as well. Are you having CUDA issues? Sorry for the delay I did not see this issue.

Thanks for your answer，I have finished training and testing on the Campus Dataset. But I have a question that: in the get_prec_recall part, you are choosing top 7 of sims to calculate pairs Fundamentalmat, right? But I find some true positive pairs are changed to true negative. Does it have influence to the final result?

OK, thanks for your help and waiting for your paper.


bazel: bump com_github_lightstep_lightstep_tracer_cpp version.

This allows us to pick up
https://github.com/lightstep/lightstep-tracer-cpp/commit/d4501f84de2d149da2a7a56c545a1c40f214db3f,
which fixes a Google internal build issue with Clang.
Risk Level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch<EMAIL_ADDRESS>

While you're at it, could you please rename @com_github_lightstep_lightstep_tracer_cpp to @com_lightstep_tracer_cpp (per their WORKSPACE file)? Bazel will eventually break because of the mismatch.

@PiotrSikora done.


How can I encrypt my password with RSA public key?

In my case I need to use rsa to encrypt password before send it to server. the public key is distributed by server, and the implement java code is below:
byte[] decoded_pubkey = Base64.decodeBase64("{THE PUBLIC KEY DISTRIBUTED BY SERVER}");

RSAPublicKey publicKey = (RSAPublicKey) KeyFactory
        .getInstance(EncodingConstants.RSA)
        .generatePublic(new X509EncodedKeySpec(decoded_pubkey)); //use X509EncodedKeySpec

Cipher cipher = Cipher.getInstance(EncodingConstants.RSA);
cipher.init(Cipher.ENCRYPT_MODE, publicKey);

String encodedPassword = Base64.encodeBase64String(cipher.doFinal("my_password".getBytes(StandardCharsets.UTF_8)));

I tried rewrite this with rust:
let pukey = "{pubkey distributed by server}";

let decoded_pub_key = base64::decode(pukey).unwrap();

//question 1
let rsa_pub_key = rsa::RSAPublicKey::from_pkcs8(&decoded_pub_key).unwrap();

let password = "my password";

//question 2 and 3
let encrypt_pass = rsa_pub_key.encrypt(RNG, PaddingScheme::new_pkcs1v15_encrypt(), password.as_bytes());

and I got this question:

Can I use  rsa::RSAPublicKey::from_pkcs8 to replace the java code X509EncodedKeySpec? It seems different but I can't find any other suitable replace code.
I don't have the param "RNG", where can I get it? or need I to generate it?
I can't find the parameter PaddingScheme in java code. How can I use it correctly?

Well, I found that below rust code works :
let decoded_pub_key = base64::decode(pukey).unwrap();
let rsa_pub_key = rsa::RSAPublicKey::from_pkcs8(&decoded_pub_key).unwrap();
let rng = &mut rand::thread_rng();
let encrypt_pass = rsa_pub_key.encrypt(rng, PaddingScheme::new_pkcs1v15_encrypt(), password.as_bytes()).unwrap();

The encrypt string which produced by the upper rust code can be decrypted correctly by the following java code:
byte[] inputByte = Base64.decodeBase64(encrypt_pass.getBytes(StandardCharsets.UTF_8));
byte[] decoded = Base64.decodeBase64(privateKey);
RSAPrivateKey priKey = (RSAPrivateKey) KeyFactory.getInstance("RSA").generatePrivate(new PKCS8EncodedKeySpec(decoded));
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.DECRYPT_MODE, priKey);
String result = new String(cipher.doFinal(inputByte));

I don't very clear about the details, but it really works for me.

Closed for no more discussion.


[Research] Custodial Service Providers for Shamir Secret Share Storage

Research Context:
We're exploring security enhancements for our web3-based wallet platform using Shamir's Secret Sharing scheme. Our primary objective is to prevent the platform from having direct access to the user's wallet key unless explicitly authorized. We're investigating the use of custodial service providers to securely store the shares generated from the secret sharing process.
Research Goals:

Please refer to the notion Doc to have a better clarity on the architecture

Identify Potential Providers: Compile a list of custodial service providers that offer secure storage solutions compatible with Shamir Secret Shares.

Evaluate Security: Assess the security measures employed by each provider to protect stored shares.

API & Integration: Analyze the API documentation and integration capabilities of each provider.

User Authorization: Determine how each provider supports user authorization mechanisms for accessing shares.

Cost Analysis: Compare the pricing models of different providers.

Compliance: Review the compliance status of each provider with relevant regulations in our jurisdiction.

Additional Considerations:

Reputation & Track Record: Research the reputation and track record of each provider in the secure storage industry.
Open Source Options: Explore if any open-source custodial solutions or libraries exist that could be adapted for our needs.

Expected Outcome:

A comprehensive report detailing the findings of our research, including a comparative analysis of potential custodial service providers.
Recommendations for the most suitable provider(s) based on our specific requirements and constraints.

We welcome any input, insights, or recommendations from the community on this research topic.
Feel free to share your research in notion docs below !
Thank you!

@axatbhardwaj can I pick this enhancement sir

@axatbhardwaj can I pick this enhancement sir

go for it , it is open for all either way
just try to ignore torus like solutions

@Ameerjafar
Hi any updates ?

@axatbhardwaj can you assign this issue to someone else

hi @axatbhardwaj ,
We can use passkey and webauth to store and encrypt keys on IPFS, encryption key would be with user(passkey) and once user login using passkey they can decrypt the key.
let me know if you want to discuss more on this

hi @axatbhardwaj , We can use passkey and webauth to store and encrypt keys on IPFS, encryption key would be with user(passkey) and once user login using passkey they can decrypt the key.
let me know if you want to discuss more on this

of course would love to

hi @axatbhardwaj , We can use passkey and webauth to store and encrypt keys on IPFS, encryption key would be with user(passkey) and once user login using passkey they can decrypt the key.
let me know if you want to discuss more on this

would love to

You could do the following thing store all encrypted shard in db.

Key 1 : General Secret Key for all users
Key 2 : Unique secert in passkey that can be access only via webauthn authentication and use it to decrypt the second shard.
Key 3 : Store encrypted key 3 (can be decrypted only by general secret) into Google Drive of user.

By this way you want be dependent on any third party and role and responsibility of the key is distributed like Key1 (platform) , Key2(user) and Key3 (combination of both platform and user).
Also allow user to regenerate new keys / such that in case of loss of passkey or loss of google drive file user can atleast get back the access.

A similar architecture is used by torus and web3auth and t4 etc...
but i think we need more clarity on the passkey implementation


access_log within if out location context

Hello,
I find the constraint for the directive access_log inside if only under location quite limitating. I'm not sure if this constraint is due thinging to the security of the product. To be clear I don't see this extraordinary difference between to be in or out location in respect of security.
I was in the need to do something similar to this:
server {
listen 80;
listen [::]:80;
if ($monitortool = 0) {
access_log  5mode-com.log combined buffer=64k flush=10m;
}
[...]
}
Adding that true I could use the if option of access_log and in my case could be great. But considering that I could have more flags indicating to me if logging is due or not I find Nginx limitaiting myself.
Hope this helps.
-Dan

The same functionality can be achieved already through conditional access logs, ie the if=condition argument of access_log directive..

Suggestion was provided. Closing this issue.


Remove from doc old starter project

About this page: https://chatie.io/wechaty/
I sugggest to remove this link: Wechaty Starter Project -> https://github.com/lijiarui/wechaty-getting-started

@realtebo Thank you for pointing this old and outdated project out!
Instead of removing the link to it, we would like to improve(rewrite) it and  get it to work with the latest Wechaty code.
Sorry for the misleading by it to you today, hope you could understand that we had moved very fast with this opensource project recently, and leave many jobs behind us.
BTW: I had just rewrite this repository and make sure it works with the last version of Wechaty now. Please have a look again at https://github.com/lijiarui/wechaty-getting-started, and feel free to let me know if you still have any issues about Wechaty.
Have a good day!

The new versions seems more linear.
I dislike the fake video link that point to the blog post, because when I am here the video is not telling me how to install, so it should be directly linked and should be placed AFTER the 'RUN' section.
Also this initial text is not so usefull because is without context

Functions as follows:
Show a QR Code for scan
Login with your user information
Log all the messages to console

You probably here should present what is the framework and what can it do.
Any way, trying to READ and FOLLOW....
I got this when npm installing
D:\wechaty-getting-started\node_modules\node-expat>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Warning: Missing input files:
D:\wechaty-getting-started\node_modules\node-expat\build\deps\libexpat\..\..\..\deps\libexpat\version.c

And this after login:
(node:12948) UnhandledPromiseRejectionWarning: Error: AssertionError [ERR_ASSERTION]: 1 == 0
    at PuppetWechat4u.puppet.on.error (D:\wechaty-getting-started\node_modules\wechaty\src\wechaty.ts:575:32)
    at PuppetWechat4u.emit (events.js:182:13)
    at PuppetWechat4u.EventEmitter.emit (domain.js:442:20)
    at PuppetWechat4u.emit (D:\wechaty-getting-started\node_modules\wechaty-puppet\src\puppet.ts:212:18)
    at Wechat.wechat4u.on (D:\wechaty-getting-started\node_modules\wechaty-puppet-wechat4u\src\puppet-wechat4u.ts:247:12)
    at Wechat.emit (events.js:182:13)
    at Wechat.EventEmitter.emit (domain.js:442:20)
    at D:\wechaty-getting-started\node_modules\wechat4u\src\wechat.js:200:14
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:12948) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:12948) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I installed docker, which temporary disallows me to run vmware workstation, just to try it in a controlled environment.
I got same error:
https://api.qrserver.com/v1/create-qr-code/?data=https%3A%2F%2Flogin.weixin.qq.com%2Fl%2FIfXEuTejXw%3D%3D&size=220x220&margin=20
(node:19) UnhandledPromiseRejectionWarning: Error: AssertionError [ERR_ASSERTION]: 1 == 0
    at PuppetWechat4u.puppet.on.error (/wechaty/src/wechaty.ts:575:32)
    at PuppetWechat4u.emit (events.js:182:13)
    at PuppetWechat4u.EventEmitter.emit (domain.js:442:20)
    at PuppetWechat4u.emit (/wechaty/node_modules/wechaty-puppet/src/puppet.ts:212:18)
    at Wechat.wechat4u.on (/wechaty/node_modules/wechaty-puppet-wechat4u/src/puppet-wechat4u.ts:243:12)
    at Wechat.emit (events.js:182:13)
    at Wechat.EventEmitter.emit (domain.js:442:20)
    at /wechaty/node_modules/wechat4u/src/wechat.js:200:14
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:19) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:19) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Hello @realtebo,
Thank you for your Prologue, I really appreciate your words on our project and ourselves.
I understand that you had many troubles when getting started with Wechaty, really sorry about that.
There are many reasons related to those issues, I'll list some of them at here:

npm install: That should work under windows, because we are using Appveyor(Win32) CI at here with a green status. However, I had not run Wechaty under windows for a long time, so I'm not sure about that. for now, at least you can check the CI setting to help you install under win32 successfully.
peer dependencies: We are splitting part of our code base to solo NPM module this week, and it's in heavy refactoring/debuging right now(with v0.17). See #1371 #1372 #1419
error after login: it seems that your account can not use Web Wechat. If you can not login via https://wx.qq.com, then you can not use Web API in Wechaty.

Summary
You need use Wechaty with Padchat Puppet. I saw you had already applied the padchat alpha testing, hope @lijiarui could be able to process your application soon.
BTW: my wechat is 918999, please feel free to add me and I might be able to support you via wechat if possible.

Thanks for your reply, and patience.
I confirm that I cannot use Web wechat. I got
<error><ret>1203</ret><message>Per salvaguardare la sicurezza del tuo account, l'accesso a WeChat via Web è stato sospeso. Puoi accedere a WeChat per Windows o a WeChat per Mac su un computer. Scarica WeChat per Windows o Mac all'indirizzo http://wechat.com.</message></error>
Which in english simply means I cannot of wechat web.
I'll wait for entering in the new alpha team.
For me it's still a lot pbscure  what "Padchat Puppet"  mean.
I know puppets system, I use in my job to keep synced about 150 windows servers configurations, updated, legacy win software deployment/update etc...
But how puppet system is related to wechat?
I was able to understand the use of headleass chrome to mimic user / browser using wechat via web. But ... ipad ? puppets ?
I'm not able to find some background info.

Compilation with actual master (#8e3a04a) works well.
Npm install warn about some c or c++ code not so safe because of a unsafe cast, but it's only a warning at it related to a third party module (nosql-leveldb).
D:\wechaty-getting-started\wechaty-getting-started\node_modules\nosql-leveldb\deps\leveldb\leveldb-1.20\db/skiplist.h(360): warning C4312: 'reinterpret_cast': conversion from 'int' to 'void *' of greater size (compiling source file leveldb-1.20\db\memtable.cc) [D:\wechaty-getting-started\wechaty-getting-started\node_modules\nosql-leveldb\deps\leveldb\leveldb.vcxproj]
  D:\wechaty-getting-started\wechaty-getting-started\node_modules\nosql-leveldb\deps\leveldb\leveldb-1.20\db/skiplist.h(337): note: while compiling class template member function 'void leveldb::SkipList<const char *,leveldb::MemTable::KeyComparator>::Insert(const Key &)'
          with
          [
              Key=const char *
          ] (compiling source file leveldb-1.20\db\memtable.cc)
  leveldb-1.20\db\memtable.cc(105): note: see reference to function template instantiation 'void leveldb::SkipList<const char *,leveldb::MemTable::KeyComparator>::Insert(const Key &)' being compiled
          with
          [
              Key=const char *
          ]

Dependencies check now tell me this; it's to fix, but it's not a stopping problem.
Chromium downloaded to D:\wechaty-getting-started\wechaty-getting-started\node_modules\puppeteer\.local-chromium\win64-564778
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.

Running, we discovere anothere missing dep. I report just for your convenience:
$ node examples/simplest-bot/bot.js
Error: Cannot find module 'wechaty-puppet-mock'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
    at Function.Module._load (internal/modules/cjs/loader.js:520:25)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (D:\wechaty-getting-started\wechaty-getting-started\node_modules\wechaty\dist\src\puppet-config.js:3:31)

Tried to fill the dependency gap
$ npm install wechaty-puppet-mock@latest wechaty-puppet@latest --save
I see now the 'cannot login' case is handled in a better way, Great !
But it's still creating some PromiseRejectionWarning
23:53:44 WARN PuppetPuppeteerBridge testBlockedMessage() error.ret=1203
23:53:44 INFO PuppetPuppeteerBridge

        Please see: Account Login Issue <https://github.com/Chatie/wechaty/issues/872>

23:53:44 ERR PuppetPuppeteerBridge init() initPage() onLoad() exception: Error: Per salvaguardare la sicurezza del tuo account, l'accesso a WeChat via Web è stato sospeso. Puoi accedere a We
Chat per Windows o a WeChat per Mac su un computer. Scarica WeChat per Windows o Mac all'indirizzo http://wechat.com.
23:54:11 WARN PuppetPuppeteer initWatchdogForPuppet() dog.on(reset) last food:[object Object], timeout:60000
(node:8108) UnhandledPromiseRejectionWarning: Error: Error: Per salvaguardare la sicurezza del tuo account, l'accesso a WeChat via Web è stato sospeso. Puoi accedere a WeChat per Windows o a
 WeChat per Mac su un computer. Scarica WeChat per Windows o Mac all'indirizzo http://wechat.com.
    at PuppetPuppeteer.puppet.on.error (D:\wechaty-getting-started\wechaty-getting-started\node_modules\wechaty\dist\src\wechaty.js:402:44)

Thank you for your patience too. Please understand that Wechaty is an opensource project with a very limited resource, so your feedback and helps will be really helpful.

what "Padchat Puppet" means.

"puppet system is related to wechat?" NO. puppet system is neither related to wechat,  nor related to Wechaty. In wechaty, the Puppet we talk about is totally different to the puppet that you used before.
the term Puppet in Wechaty is a name that we had picked up to describe part of our system: Puppet is an Abstract Class for implement Plugins, the plugins is the component that helps Wechaty to control the Wechat, that's the reason we call it puppet.
the plugins is named XXXPuppet, like PuppetPuppeteer is using the chrome puppeteer to control the WeChat Web API via a chrome browser, PuppetPadchat is using the WebSocket protocol to connect with a Protocol Server for controling an iPad program.
For deeper understand of the Puppet in Wechaty, you can read its source code if you like at https://github.com/Chatie/wechaty-puppet/blob/master/src/puppet.ts

the dependency gap - Sorry for that. As I mentioned before, we are spliting the source code of Wechaty into many single npm modules those days, bugs were produced everyday, including this one: I had forgoten to add them to the dependencies. It had been fixed in the latest code from master branch.
creating some PromiseRejectionWarning - You need to register a error handle for Wechaty to prevent that, like: bot.on('error', yourErrorHandler). Like a stream, Wechaty will throw unhandled error when you have no error handler(fair enough right?). However, there do have many cases that Wechaty has many inner PromiseRejection, if you meet them and know how to reproduce, please file an issue.

@realtebo Hello, do you have more questions about this issue?
Could we close this issue for now?


Ag grid v32 floating filter active state

Description
v32 changed floating filter's "active" state to a toggle button inlined with the input (previously its indicated by an icon on the header cell)
https://www.ag-grid.com/javascript-data-grid/floating-filters/
.ag-filter-active is applied to the button, will need Salt support.
Expected behavior
Visual indicator to show the filter is on
Alternatives and examples

Context
Upgrade ag grid from v31 to v32
Are you a JPMorgan Chase & Co. employee?

[X] I am an employee of JPMorgan Chase & Co.

Latte Goal: v32 upgrade release candidate

Oct 17 - send snapshot for MTK to test, also testing in Stackblitz for v31 backwards compatibility

Oct 21 : Ben to review previews in PR

Oct 23 - MTK testing snapshot, visual regression changes are mostly expected code change

moving into Lungo pending @bhoppers2008 review

Met with Ben. We want combined approaches between PRs

Use toggle button active style
Use different filter icons, solid for active and outline for default state
Decrease size of HD compact "fake" toggle button to align with base height 16px

To be released in #4418

Done and released.

https://github.com/jpmorganchase/salt-ds/releases/tag/%40salt-ds%2Fag-grid-theme%402.2.0


Add a test for issue 1211, showing it's no longer an issue.

Since rustfmt has moved away from syntex the overflow seen in https://github.com/rust-lang-nursery/rustfmt/issues/1211 is no longer a problem. This commit adds a test to verify that.
Closes #1211.

Thank you!


Game window and all icons/messages size is absolute

Description of the Issue
All game objects- logo, insctructions, game window appear fixed size. Can be changed to relative size. For, bigger resolution monitors, game looks same small size.
Steps to Reproduce the Issue

Open game page on high resolution monitor, or change resolution of monitor.
Game appears of same size irrespective of monitor resolution

Expected Behavior
Game objects(width and height) should be changed based on monitor resolution.
Actual Behavior
Game objects are of same size.

Up for grab :)


Next 13 app dir - Error: Dynamic server usage: force-dynamic

What version of this package are you using?
    "next-translate": "^2.3.1",
    "next-translate-plugin": "^2.3.0",

What operating system, Node.js, and npm version?
npm v18.16
pnpm v8.6.3

What happened?

I followed exactly app-dir examples and facing this in every page navigate

Extra information
module.exports = {
  locales: ["default", "en", "ms"],
  defaultLocale: "default",
  localeDetection: false,
  pages: {
    "*": ["common"],
    "/[lang]": ["common"],
  },
}

Interesting... We are using export const dynamic = 'force-dynamic' to allow dynamic rendering inside (/[lang]). However, I never experimented this issue before. Would be great if you change a reproducible example to investigate why this is happening 🙏
BTW; for now you can change this dynamic behavior, by adding this to the page export const dynamic = 'auto'.

Please @darr1s try this prerelease and give some feedback 🙏

Next-translate: 2.4.0-canary.4
Next-translate-plugin: 2.4.0-canary.4

Thanks! I've upgraded to canary 4 and dynamic seems to throw a new error from the lib:
Error: Dynamic server usage: searchParams.lang

Source
src/app/[lang]/login/page.tsx (80:20) @ searchParams

  78 | const config = {
  79 |   ...__i18nConfig,
> 80 |   locale: props.searchParams?.lang ?? props.params?.lang ?? __i18nConfig.defaultLocale,
     |                ^
  81 |   loaderName: 'server /page',
  82 |   pathname: '/[lang]/login/',
  83 |   loadLocaleFrom: typeof __i18nConfig.loadLocaleFrom === 'function' ? __i18nConfig.loadLocaleFrom : (l, n) => import(`@next-translate-root/locales/${l}/${n}`).then(m => m.default),

@darr1s thanks for the feedback

I guess is related with this Next.js issue https://github.com/vercel/next.js/issues/49182. But in reality searchParams is not necessary to use it, I changed the order prioritizing params, in this way is not necessary to add the export const dynamic = 'force-dynamic' that they comment in the Next.js issue.

I hope now works in your case:

Next-translate: 2.4.0-canary.4
Next-translate-plugin: 2.4.0-canary.5

Any news @darr1s ? 😊

Any news @darr1s ? 😊

Hey! Pardon I forgot to close the issue. It worked, but I'm faced with some issue where Next 13 will hang the browser to death (I'm thinking is the middleware prefetch or redirection) and not related to this issue anymore.
Thanks for the help!

Ok I'll close this issue, if you found that is related with next-translate feel free to report another issue. Thanks for your feedback ☺


[camera_android] Add support to manipulate focus distance and focus mode manually

Use case
in https://pub.dev/packages/camera
I need to manipulate focus distance manually
specifically this feature --
set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF)
set(CaptureRequest.LENS_FOCUS_DISTANCE, value)
these methods are available on android camera2 for CaptureRequest.Builder
thanks in advance

this my try to change LENS_FOCUS_DISTANCE manually

Hi @abo-el30la
It looks like you are filing a proposal/feature request. Please try to update the issue by the template so it will have better readability.

Issue Template
<!-- Thank you for using Flutter!

     If you are looking for support, please check out our documentation
     or consider asking a question on Stack Overflow:
      * https://flutter.dev/
      * https://api.flutter.dev/
      * https://stackoverflow.com/questions/tagged/flutter?sort=frequent

     If you have found a bug or if our documentation doesn't have an answer
     to what you're looking for, then fill out the template below. Please read
     our guide to filing a bug first: https://flutter.dev/docs/resources/bug-reports
-->

## Use case

<!--
     Please tell us the problem you are running into that led to you wanting
     a new feature.

     Is your feature request related to a problem? Please give a clear and
     concise description of what the problem is.

     Describe the alternative solutions you've considered. Is there a package
     on pub.dev/flutter that already solves this?
-->

## Proposal

<!--
     Briefly but precisely describe what you would like Flutter to be able to do.

     Consider attaching something showing what you are imagining:
      * images
      * videos
      * code samples

     Does this have to be provided by Flutter directly, or can it be provided
     by a package on pub.dev/flutter? If so, maybe consider implementing and
     publishing such a package rather than filing a bug.
-->

Thank you!

Proposal
in https://pub.dev/packages/camera
I need to manipulate focus distance manually
specifically this feature --
set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF)
set(CaptureRequest.LENS_FOCUS_DISTANCE, value)
these methods are available on android camera2 for CaptureRequest.Builder
thanks in advance

Hi @abo-el30la
Please don't repeat the same description in the comments. I propose a new description as below:

Usecase
I need to manipulate focus distance and focus mode manually. These methods are available on android camera2 for CaptureRequest.Builder:
set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF)
set(CaptureRequest.LENS_FOCUS_DISTANCE, value)

Proposal
Support this feature on camera_android

If you agree/disagree about this, please write in the comments. Thanks!

yes , I agree

Thank you for confirming. Labeling this issue as a feature request.

is this feature will be available soon?

Please see https://github.com/flutter/flutter/wiki/Issue-hygiene#when-will-my-bug-be-fixed.


⚠ Bofrost.de has degraded performance

In fc82086, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 839 ms

Resolved: Bofrost.de performance has improved in 412b3eb after 309 days, 7 hours, 49 minutes.


Android: Modal announcements

Requires API Proposal
This issue may require a new API, or a change to an existing API. An API proposal should be added and discussed before proceeding with implementation. The API proposal can be added in the comments of this issue or linked as a separate issue.
Description
On Android, when new Windows appear, their title is announced.  If no title is set, it will treat the first visible item as a title, and instead announce it, moving focus automatically to the second element. This is not always desired, so we should provide a way to set a specific title to be announced instead.
This issue specifically applies to the  component (https://reactnative.dev/docs/modal), which does not have a way to specify a title. Other Window types, such as Alert's, work correctly as their only content is treated as the title.
React Native version:
v0.63
Expected Behavior
Upon appearance, 's should announce a specified title, and focus should move into them, landing on the first visible element.  If no title is specified, the existing behavior should apply, which treats the first visible element as the title, with focus moving to the second visible element.
Android Details
In Android Windows such as "Dialog" have the ability to set a title via a "setTitle()" method (https://developer.android.com/reference/android/app/Dialog#setTitle(java.lang.CharSequence)).
In React Native, when we create a new Dialog for the  component, we do not set this title (RN source here:
https://github.com/facebook/react-native/blob/5348a98207dcff1686b2964d51572c3b226663ab/ReactAndroid/src/main/java/com/facebook/react/views/modal/ReactModalHostView.java#L265)
We need to set this title for the accessibility system to pick it up.
If you want the title to only be picked up by the accessibility system, but not visible displayed, you only need to set the title on the Window attributes, which you should be able to do by calling dialog.getWindow().getAttributes().setTitle("my title").  This is exactly what happens when you call setTitle() on the Dialog itself in the Android source.
Android source here: https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/android/app/Dialog.java;l=611

Modal is just a container so you can build your own UI. Or you can use third-party ready made modals.

The intent of this issue is for React Native itself to perform these actions.

@kacieb, @lunaleaps, @nadiia, do we want RN's  dialog to have a way to visibly display a title as well as have one set for accessibility? Right now, it relies on a user passing content into it that will act as a title, but this content is not technically considered a "title" by Android at an OS level, which leads to this problem.
We can set a title only for accessibility, or we could have the option to set a title both visibly and for accessibility, or we could do both.

Just wanted to check - it sounds like "title" is different from "accessibilityLabel" (or contentDescription on Android)? It seems reasonable to me to add a new prop to Modal to allow people to pass in a title for accessibility purposes. This would require both Android and iOS work.
I think this component is generic enough that we don't want to render the title for them, though (since you can put anything you want in this modal) - the prop should only be for accessibility. We should be very descriptive about when to use it in our documentation about the prop. The only issue I see with adding a prop only for accessibility is it wouldn't move the focus correctly to the first "non-title" element.
This component could honestly also use a revamp in general, I noticed quite a few deprecated lifecycle methods when looking at it, (and there may be better solutions elsewhere in open source), so I don't think this issue is high priority.

Just wanted to check - it sounds like "title" is different from "accessibilityLabel" (or contentDescription on Android)? It seems reasonable to me to add a new prop to Modal to allow people to pass in a title for accessibility purposes. This would require both Android and iOS work.

Yeah, on Android a Window (which a dialog is) doesn't have the normal View properties like contentDescription, etc. A title is meant to be visibly displayed at the top of the Window and used to let the user know what the purpose of this window is.
When one is set Talkback is also smart enough to know to announce this when the window first appears, and not to focus on it directly.  It's not a property meant specifically for accessibility though, but there are implications if one isn't set, as Talkback will still read something when the Window appears, but since it's not something we define, the behavior may not be what a developer expects.
As a fun aside, the same general rule applies to activities on Android, which can also have titles set, but if you don't set them, it will just read your apps name instead of trying to find a title in the content. If you've used the an app and heard Talkback say the app name over and over whenever you open some new screen, it probably means you forgot to add an activity title.

@blavalla I included an API Proposal in the Draft PR https://github.com/facebook/react-native/pull/34969 Thanks
Video of the Functionality on Android https://www.icloud.com/iclouddrive/0fdBl7MCKgQ3d0vMeRR-DDVuA#modal_title_android


[BuildPlan] Add a way to traverse the graph and compute destinations for all modules

Motivation:
This is going to be used by sourcekit-lsp to build a graph/dictionary
of all the targets and depths they appear during package loading and
replaces the need for topological sort and buildTriple.
Modifications:

Introduces a new depthFirstSearch variant which traverses all of the nodes in the graph.
Adds a new method to BuildPlan - traverseModules which goes over the graph, computes destinations and returns modules with their parents and depths they appear at (the method looks though products).

Result:
This make it possible to switch sourcekit-lsp away from using topological sort and relying on buildTriple.
Paired with https://github.com/swiftlang/sourcekit-lsp/pull/1621

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test macOS platform

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test macOS platform

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test macOS platform

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test Linux platform

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test Windows platform

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test Windows platform

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test

https://github.com/swiftlang/sourcekit-lsp/pull/1621
@swift-ci please test Windows platform


Add sidemenu callback

Hi!
This is very cool library and it was integrated into my application. That's great!
However, I would like to know when the side menu was closed or opened. For ex: The left menu contain so feature, that I dont want user to access right now, so everytime user try to open the menu, I want to notify user about that and recommend user to upgrade to paid version...
That is very handy to know when user close/open the side menu or event change side or center view. That help application can manage side menu completely
Please think about that and if it could available in some update later, that would be awesome!!
Thank

@pvn-anh-pham
Thank you for the appreciation. You can handle this situation with the FAPanelStateDelegate. It gives you access when the left, right and centre panel has became active or not.


Add support for hidden roles

In the bot config, add support for hidden assignable roles. Hidden roles will not show up in the /roles command or be suggested by autocomplete for /role. This will let us provide secret roles for people to discover by trying.
The config change is super easy. Just add a hidden flag to the roles JSON format.

implemented in eedbab375e9f482c62e9c605ce139e221e7be547


Supporting action output

👋 Hi! My team has been using this action, and we were wondering if it's possible to get the summary output from the action to use in subsequent workflow steps. A fake example:

We'd like to have this summary output available as output from the action itself, so that we can do things like post it as a comment on the PR that corresponds to the workflow run, notify people in Slack, etc. Is that supported today and I'm just missing it? If not, I would be happy to put together a pull request to support this.

Hi @davidmally-at! I don't think we have anything in place to support the use case you're proposing, a PR would be extremely welcome! 🙇

J

I'd also support this feature request.
The summary used for a PR comment is compiled. All that is left to do is output that from the workflow so that it could be picked up by later steps.

If that's all that's desired, then it'd be a pretty easy thing to do right here:
https://github.com/actions/dependency-review-action/blob/80f10bf419f34980065523f5efca7ebed17576aa/src/comment-pr.ts#L16-L23

This functionality is now available in the latest release of the Action (4.1.2) thanks to @jsoref. You can find an example here of how to use the output. Closing this issue.


[Suggestion] Animated tiles with the Coin or Coinblock properties will display as an animated tile rather than using the default coin/coinblock textures

Not sure how this would affect animated tiles that are not coins/coinblocks for all of their frames, though.

I know that animated tiles that are coins don't work anyway, so that would be a good fix anyhow.


[iOS] Fix AboutView content padding

Idea Description

Currently, AboutView 's content paddings (for example, description padding) are not correct.

What is DroidKaigi? Label
Description
Tiles (Access, Staffs, Privacy Policy)

Reference images and links

Current
Expected

I would like to challenge this.

💪💪💪💪
assigned!!!

I'm sorry, I wrote some parts that didn't need to be fixed due to my misunderstanding 🙇🏼
Here's what needs fixing:

Tiles (Access, Staffs, Privacy Policy)


fix HTTP proxying

What do these changes do?

fixes HTTP proxy support.
adds functional tests for proxies (via Mitmdump)

Are there changes in behavior for the user?

Fixes regression, brings back intended behavior, users will be able to use HTTP proxies without any errors.
Related issue number

#1413
Checklist

[x] I think the code is well written
[x] Unit tests for the changes exist
[ ] Documentation reflects the changes
[ ] Add yourself to CONTRIBUTORS.txt

The format is <Name> <Surname>.
Please keep alphabetical order, the file is sorted by names.

[ ] Add a new entry to CHANGES.rst

Choose any open position to avoid merge conflicts with other PRs.
Add a link to the issue you are fixing (if any) using #issue_number format at the end of changelog message. Use Pull Request number if there are no issues for PR or PR covers the issue only partially.

Current coverage is 98.50% (diff: 100%)

Merging #1415 into master will decrease coverage by 0.37%

@@             master      #1415   diff @@
==========================================
  Files            30         30
  Lines          6900       6902     +2
  Methods           0          0
  Messages          0          0
  Branches       1143       1144     +1
==========================================
- Hits           6823       6799    -24
- Misses           33         39     +6
- Partials         44         64    +20

Powered by Codecov. Last update b65ecf4...ac8b43f

Long story short
I tried this patch and found the related bug, it appears after a redirect, I think I should put it here until it's in production
Expected behaviour
http://httpbin.org/redirect/6 (302) -> http://httpbin.org/get (200)
Actual behaviour
Traceback (most recent call last):
  File "/home/user/Projects/test_beh.py", line 14, in <module>
    loop.run_until_complete(go())
  File "/usr/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
    return future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/home/user/Projects/test_beh.py", line 8, in go
    proxy="http://<IP_ADDRESS>:29842") as resp:
  File "/home/user/envs/test_env/lib/python3.5/site-packages/aiohttp/client.py", line 529, in __aenter__
    self._resp = yield from self._coro
  File "/home/user/envs/test_env/lib/python3.5/site-packages/aiohttp/client.py", line 168, in _request
    resp = req.send(conn.writer, conn.reader)
  File "/home/user/envs/test_env/lib/python3.5/site-packages/aiohttp/client_reqrep.py", line 442, in send
    path = self.path
AttributeError: 'ClientRequest' object has no attribute 'path'

Steps to reproduce
import asyncio
import aiohttp

async def go():
    async with aiohttp.ClientSession() as session:
        async with session.get("http://httpbin.org/redirect/6",
                               proxy="http://<IP_ADDRESS>:80") as resp:
            print(resp.status)

loop = asyncio.get_event_loop()
loop.run_until_complete(go())
loop.close()

Your environment
python 3.5.2
aiohttp 1.1.5 with this patch
Other
random proxy list http://hideme.ru/proxy-list/?ports=80#list

There is applied #1421
Please check with a fresh master.
I have nothing against mitmproxy (or other real proxy server, maybe https://github.com/hawkowl/rproxy is also a good candidate for running twisted based server in the same loop -- I don't know).

I believe it should be closed


WFLY-4389: Upgrade HAL to 2.6.5.Final

https://issues.jboss.org/browse/WFLY-4389

Closing this, as this is based on my original PR from last week


[Discussion]: New $ and $$ type inference makes code harder to read

Bug description
v15 tries to infer types for simple selectors like 'a' or 'tr'. It removed the ability to provide explicit types via generics and requires casting for non-trivial selectors.
I would argue that most code doesn't use trivial selectors and provides a nested path or classes instead. Developers often know the return type already and the new inference doesn't allow them to provide these types.
// v14
await page.$<HTMLAnchorElement>('a')
await page.$<HTMLAnchorElement>('.class > a')

// v15
await page.$('a')
(await page.$('.class > a')) as ElementHandle<HTMLAnchorElement> | null

Simply put, while the simple cases are now shorter, the more complex cases are now longer/harder to read.
I propose that puppeteer reinstates the ability to specify a generic type in addition to the current smart inference.
Puppeteer version
v15.1.1
Node.js version
v16.15.1
npm version
Yarn 3.2.1
What operating system are you seeing the problem on?
macOS
Relevant log output
No response

Actually, I agree with this. Having both would be great and it looks like we overlooked the impact on the more complex selectors. @jrandolf WDYT? I am not sure if it's possible to have both ways supported.

Hey, thanks for bring this up! While making the PR, I wanted to discuss the reasoning behind this decision in detail, but for brevity, I chose to just leave it out.
There are two core problems with allowing users to set their own return type (and one extra side note):

A malicious/unknowing user can set the return type to something completely wrong and get away with it. This is a result of the ReturnType type parameter not being used in any way for type inference. For example, the v15 method uses the selector argument parameter to deduce the return type.
Even if the user is correct, this doesn't imply they always will be. For example, in your example, suppose you just wrote .class instead of .class > a. At the moment you code this, you may know it's correct, but a year later, suppose someone else changed the tag name of some .class element. Then the code has potential to break at the runtime level while still being correct at the type level.

[Side note] @ahejlsberg, the lead architect of TypeScript, has also commented about this (here) remarking this is an anti-pattern.

From these problems, you can deduce more problems such as maintainers (us) getting issues due to some library lying to their users about a runtime problem when it was in fact issue (2) in play.
Now all is not lost. TypeScript has some great features such as template types. For example, we could do the following:
function $<Selector extends keyof HTMLElementTagNameMap>(
    selector: `${string} ${Selector}`
): Promise<ElementHandle<HTMLElementTagNameMap[Selector]> | null>;

This would deduce your .class > a to an HTMLAnchorElement. As a side note, this would also strength issue (2) since any type you put in a hypothetical ReturnType will always have a chance to be wrong since any selectors not conforming to ${string} ${Selector} will based on a non-tagname based one.

Got it. I do agree that page.$<HTMLAnchorElement>(...) is essentially a cast in disguise. I guess the new approach is to place the decision to unsafe cast into the developer's hand, instead of hiding it. I was also under the assumption that complex selectors could never be inferred in the future, but this is obviously proving me wrong.
My main concern is that in the meantime, this example:
// Before
await page.$<HTMLAnchorElement>('.class > a')

// After
(await page.$('.class > a')) as ElementHandle<HTMLAnchorElement> | null

requires developers to look at the interface of $, write ElementHandle, provide the element type, add nullability, all while running into the risk of missing some of those.
I wonder if there's a middle ground where either:

$ would accept another function parameter – an element name – and fail if the handle doesn't have the proper type:

// Validates that the selector returns an anchor, then returns it casted
const anchor = await page.$('.class > a', { element: 'a' })

puppeteer would expose a function that validates, then casts an element handle.

const handle = await page.$('.class > a')
const anchor = await validateHandle(handle, 'a')

But if you do have typed query selectors on the roadmap, feel free to ignore all of this, I'll cast in the meantime 👍

Thanks for understanding. We'll consider the proposed solutions; they are more feasible since they implement runtime checks, however I'm hard-pressed those solutions should belong in puppeteer. @OrKoN WDYT? We could implement something like
export async function assertElementHandle<T extends keyof HTMLElementTagNameMap>(handle: ElementHandle, tagName: T): handle is ElementHandle<HTMLElementTagNameMap[T]> {
  return await handle.evaluate((e, tagName) => {
    return !!(e instanceof Element && e.tagName.match(new RegExp(tagName, "i")));
  }, tagName);
}


Public Dockerfile

What does this PR do ?
Adds a public facing dockerfile
Changelog

Please update the CHANGELOG.md under next version with high level changes in this PR.

Usage

You can potentially add a usage example below

docker image build --network=host -t nemo-aligner:nemo-1.21.0-mlm-0830-TE-1.1 .

Before your PR is "Ready for review"
Pre checks:

[X] Make sure you read and followed Contributor guidelines
[ ] Did you write any new necessary tests?
[ ] Did you add or update any necessary documentation? Make sure to also update the NeMo Framework User Guide which contains the tutorials

Checklist when contributing a new algorithm

[ ] Does the trainer resume and restore model state all states?
[ ] Does the trainer support all parallelism techniques(PP, TP, DP)?
[ ] Does the trainer support max_steps=-1 and validation?
[ ] Does the trainer only call APIs defined in alignable_interface.py?
[ ] Does the trainer have proper logging?

Additional Information

Related to # (issue)

I have updated the Docker file to use the latest NeMo 1.22.0 paired with the latest main branch of Megatron-LM and TransformerEngine 1.1. Confirmed working for DPO training and generation.

If you use --no-deps, then it should technically just be: RUN pip install --no-deps<EMAIL_ADDRESS>I will commit these changes later today
On Thu, 21 Dec 2023 at 13:07, Yi Dong @.***> wrote:

@.**** commented on this pull request.
In Dockerfile
https://github.com/NVIDIA/NeMo-Aligner/pull/47#discussion_r1434514464:

+WORKDIR /opt/NeMo
+# fixes a bug in NeMo when used with latest apex
+RUN sed -i 's/_fast_layer_norm(x, self.weight + 1, self.bias, self.epsilon)/_fast_layer_norm(x, self.weight + 1, self.bias, self.epsilon, False)/g' nemo/collections/nlp/modules/common/megatron/layer_norm_1p.py
+RUN sed -i 's/shutil.rmtree(ckpt_to_dir(filepath))/shutil.rmtree(ckpt_to_dir(filepath), ignore_errors=True)/g' nemo/collections/nlp/parts/nlp_overrides.py
+RUN sed -i 's/[all]/[nlp]/g' reinstall.sh
+RUN rm -rf .git && ./reinstall.sh

+# MLM
+RUN pip uninstall -y megatron-core
+RUN git clone -b core_r0.4.0 https://github.com/NVIDIA/Megatron-LM.git
+RUN cd Megatron-LM && pip install -e .
+
+WORKDIR /opt
+
+# install the latest NeMo-Aligner
+RUN pip install --upgrade-strategy only-if-needed @.***
yes --no-deps works too. I tried it as the following
MLM
RUN pip uninstall -y megatron-core
RUN git clone -b core_r0.4.0 https://github.com/NVIDIA/Megatron-LM.git
RUN cd Megatron-LM && pip install -e .
WORKDIR /opt
install the latest NeMo-Aligner
RUN pip install  --no-deps --upgrade-strategy only-if-needed @.***
—
Reply to this email directly, view it on GitHub
https://github.com/NVIDIA/NeMo-Aligner/pull/47#discussion_r1434514464,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGFLGNDBIMGO5IWCEYSUIILYKSQIRAVCNFSM6AAAAABAILV2EGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTOOJTGY3TIOBQGY
.
You are receiving this because you authored the thread.Message ID:
@.***>

this dockerfile works for me. are we ready to merge it in?

I would be happy to merge it, but we need @gshennvm to confirm as well

looking good! after 2 more minor changes i think we can merge


[Event Request] Codeunit 333 Issue 1339 Not Completely Implemented

According to #1339 (shipped) I should find new events to extend the document grouping by customized fields in D365BC RTM. But some are still missing:
Function Code
[...]
  SETCURRENTKEY(
    "Worksheet Template Name","Journal Batch Name","Vendor No.",
    "Sell-to Customer No.","Ship-to Code","Order Address Code","Currency Code",
    "Ref. Order Type","Ref. Order Status","Ref. Order No.",
    "Location Code","Transfer-from Code","Purchasing Code");
  // >> NEW
  OnBeforeFindReqLines(VAR ReqLine);
  // << NEW

  IF FIND('-') THEN
[...]

From https://github.com/Microsoft/AL/issues/1339#issuecomment-361897790 (Jan 31)

Function  Code (end)
Code refactoring for new event OnCheckNewNameNecessary

Hello,
is here something new? We also have the Event Request OnBeforeFindReqLines to change the existing Keys (post 1) and FinalizeOrderHeader (post 3).
Post 4 seems to be solved.

@SKeiler
OnBeforeFindReqLines has been solved in 14.05 using OnBeforeSetReqLineSortingKey:

FinalizeOrderHeader is also solved, using OnFinalizeOrderHeaderOnAfterSetFiltersForNonRecurringReqLine

@AlexanderYakunin  The last open issue is this request part:
Function InsertPurchOrderLine

What is the current state?

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


❓ Simultaneously record video or take photos from both rear and front cameras

Question
My goal is to have both rear and ten cameras recording or taking photos at the same time, just like in the bereal app
What I tried
No response
VisionCamera Version
3.6.6
Additional information

[ ] I am using Expo
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar questions in the issues page as well as in the discussions page and found none.

Hey - I'm not sure if this is currently possible, but you can try to just render two <Camera> Views and see what happens:
<Camera device={backCamera} ... />
<Camera device={frontCamera} ... />

Hey - I'm not sure if this is currently possible, but you can try to just render two <Camera> Views and see what happens:

<Camera device={backCamera} ... />

<Camera device={frontCamera} ... />

I tried this but one worked and the other didn't

Well then yea this does not currently work and is quite complex to implement. Did you test on both iOS and Android? I thought it would work

Well then yea this does not currently work and is quite complex to implement. Did you test on both iOS and Android? I thought it would work

Honestly, I didn't try the android part after I saw that it didn't work on the ios part because it was necessary for me on both sides.

Well yea, I think this is currently not possible. It would require quite a large refactor of the codebase as on iOS I would need to use a single AVCaptureSession and attach two input Cameras to that - quite a complex thing to do.
Might significantly change the API as well:
const device = useCameraDevice('back')
const camera = useCamera(device)

return <Camera camera={camera} isActive={true} />

and for multi:
const front = useCameraDevice('front')
const back = useCameraDevice('back')
const camera = useCamera([back, front])

return <Camera camera={camera} isActive={true} />

If you want this, consider contacting me thru my agency to figure out a way to collaborate on this.

in theory, could you use 2 libraries to record? For example, use this one for back camera and another one for front camera? Or do they basically use the same iOS API and would conflict with each other?

No it is the same iOS API.


[Event Request] Report 202 Sales Document - Test - OnAfterGetRecordCheckIsHandled

Hi Team,
In Report 202 we need possibility to have our additional checks . Having event with handling in the trigger OnAfterGetRecord, would be a perfect solution :

[IntegrationEvent(false, false)]
local procedure OnAfterGetRecordCheckIsHandled(var SalesLine: Record "Sales Line"; var GLAcc: Record "G/L Account"; var IsHandled: Boolean; OrigMaxLineNo: Integer)

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


DRILL-5273: CompliantTextReader excessive memory use

DRILL-5273
CompliantTextReader exhausts 4 GB memory when reading 5000 small files
Please see JIRA for details of problem and fix.

+1


[ConstraintSystem] De-duplicate storage of constraint system elements

It's always been the case that partial solutions introduce
some storage duplication when applied back to the constraint
system to form a more complete solution with outer context,
but the constraint systems used to be small before
introduction of result builders (and now multi-statement
inference), which make the duplication more visible.

@swift-ci please test source compatibility

@swift-ci please clean test Windows platform


Update series long descriptions and rename "serie" to "series"

closes #218
closes #217
closes #220
related #222

The changes look reasonable, let me test against my data and update the ticket information that this originated from.

Ping @mico for fixes.

@opoudjis @ronaldtse the aim of this pull request is only to fix #217
I had no intention here of fixing anything else from https://github.com/metanorma/metanorma-nist/issues/392


Merge rel/2.0.0 => dev

@HaoK,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


fix deprecated translation

fix deprecated translation

/assign @sataqiu

Thanks for the good catch.
/lgtm

/lgtm cancel

/lgtm
/approve


✨ Clarify what the intended way to use the yarn action is

What are you trying to do?
In a dagger plan, if you include two Yarn actions and you don't specify a project field (which is optional), that will cause undesired effect in caching since both actions will share the same cache id as the project field is used here https://github.com/dagger/dagger/blob/main/pkg/universe.dagger.io/yarn/yarn.cue#L134 and here https://github.com/dagger/dagger/blob/main/pkg/universe.dagger.io/yarn/yarn.cue#L141 respectively.
An evident proposal to  improve this would be to make the project field required and add  proper documentation on the action module to clarify this behavior.
cc @shykes @rawkode @helderco
thoughts?
Why is this important to you?
Avoid surprises when using the Yarn action without specifying a project nam.
How are you currently working around this?
Intentionally setting different project ID's for both actions

I notice that each dagger.#FS has an ID. Would this be unique enough to use as a default value for the yarn project name?

No, that's a hidden value only accessible to the engine.

I think it's fine to make it mandatory. Making the field optional is less friction for getting started, but given the absence of API reference and proper package documentation in general, there's no guard rails when you add a second yarn build and caching starts to break. So, adding a little friction upfront seems acceptable.
Questions & notes:

I'm assuming that the caching issue caused by sharing an ID is actually bad? Like possible data corruption, deadlocks etc? I'm asking because, if the symptom is a slightly less efficient build for example, then that would change the nature of the tradeoff. This depends on how yarn implements its caching. cc @marcosnils

From first-hand experience, this ID-based model for managing cache directories is confusing and a common source of problems. The issue is that it's a completely different model than for other directories. To share a regular directory for copy or mount, you use a special type dagger.#FS, but to share a cache directory you use an ID string? I have seen several users thoroughly confused by that, and I don't blame them. I know it maps directly to how buildkit handles cache dirs... But I think we need to rethink this part of the API for hypothetical future API redesigns... cc @aluzzardi @sipsma @helderco 😇

I'm assuming that the caching issue caused by sharing an ID is actually bad? Like possible data corruption, deadlocks etc?

On David's case the issue is that he has multiple projects using the same Yarn definition and dagger would cache the initial run and then it will re-use the same cache incorrectly for the second project since he wasn't changing any inputs on the action.
@rawkode  mind sharing how you were defining your actions so Solomon has a better idea of what I'm trying to achieve? I don't have the Dagger Cloud URL's with me now :pray:

On David's case the issue is that he has multiple projects using the same Yarn definition and dagger would cache the initial run and then it will re-use the same cache incorrectly for the second project since he wasn't changing any inputs on the action.

By "re-use the same cache", do you mean "cache the operation", or "mount the same cache directory"?

closing until universe decisions are clear here: https://github.com/dagger/dagger/issues/3206


Locale issue, need to install glibc-langpack-* package

using stack to build a haskell program inside Fedora toolbox, I got a strange error that I could track down to a locale issue. There were other issues such as less not showing up UTF-8 byte sequences. Running locale showed up errors:
$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=fr_FR.UTF-8
LC_CTYPE="fr_FR.UTF-8"
LC_NUMERIC="fr_FR.UTF-8"
LC_TIME="fr_FR.UTF-8"
LC_COLLATE="fr_FR.UTF-8"
LC_MONETARY="fr_FR.UTF-8"
LC_MESSAGES="fr_FR.UTF-8"
LC_PAPER="fr_FR.UTF-8"
LC_NAME="fr_FR.UTF-8"
LC_ADDRESS="fr_FR.UTF-8"
LC_TELEPHONE="fr_FR.UTF-8"
LC_MEASUREMENT="fr_FR.UTF-8"
LC_IDENTIFICATION="fr_FR.UTF-8"
LC_ALL=

Installing the glibc-langpack-fr package (in my case) solved the issue. That should probably be performed when the toolbox is created the first time. Or it could be part of the documentation.

Yes, I have heard of similar oddity reported by Czech users too, but since locale seemed to work fine for variants of en (not just en_US), I forgot about it.
Do you know how the glibc-langpack-fr package gets installed on your Fedora host?

Thanks to @halfline I understand this a little better.
It's likely that replacing glibc-minimal-langpack with glibc-all-langpacks will solve the most pressing issues:
# dnf -y swap glibc-minimal-langpack glibc-all-langpacks

I think that a complete fix would also involve removing /etc/rpm/macros.image-language-conf and reinstalling all packages with missing translations. That would involve something similar to #55, which undoes the effects of tsflags=nodocs.

It would also be helpful to have a concrete test case that I can use to validate my experiments.

A simpler fix or workaround for now might be just to run podman with LANG=C.utf8.
That should fix most of the current problems I reckon.

Is this already fixed?

I don't know if this is following up this issue or not, but in a recent toolbox, the locale is no longer preserved from the host. now I get the en_US.UTF-8 locale, no locale error, and the same vim issues as in #14 (except not running inside tmux)

Currently toolbox only installs glibc-langpack-en by default.
Maybe you can try to install glibc-langpack-fr into your toolbox.

In latest fedora:30 (and fedora:rawhide) glibc-langpack-en was dropped.
eg this affects https://bodhi.fedoraproject.org/updates/FEDORA-CONTAINER-2019-724ac61633
I still personally think defaulting toolbox to use C.utf8 is okay when the user's locale not available.
Or maybe it is better to install glibc-langpack-XX as Mildred suggested.

Okay it seems to be covered by https://bugzilla.redhat.com/show_bug.cgi?id=1727489

Can confirm that installing glibc-langpack-en or glibc-alll-langpacks fixes this issue. Havee in mind that the later weigth about 200mb which is half's the container weigth.

Running $ toolbox run locale on the host yields:
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
(...)

It seems to inherent the host's locale, which is not the case when running $ locale within a containerized shell session:
$ toolbox enter
⬢$ locale
LANG=C.UTF-8
LC_CTYPE="C.UTF-8"
(...)

While I am not certain whether it is a separate issue, installing glibc-langpack-en or glibc-alll-langpacks fixes it for me.


Test Issue

Created issue to test if we are able to track this internally.

Hello, world!

Closing now...


Forward port from Ember: Indicate if VPCs are private or public

When creating Cloud Template in RKE in AWS it shows if VPC is private or public. In RKE2 provisioning it only lists VPC ID's so it's not clear if VPC is public or private.

Closing due to age - please re-open if this is a feature you would still like.


Typo in Windows Insider Release docs

In page https://docs.microsoft.com/en-us/hololens/hololens-insider?source=docs#windows-insider-release-notes
Under Windows Insider Release Notes
there is a typo in the word "available" & features
As of our Windows Holographic May 2020 Update release all of our release preview feautres [features] are now generally avalible [available]!
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: dafbb989-d1f4-0411-81ce-f761e83bdeba
Version Independent ID: e8fdf590-c7f0-6b07-9407-33bdf2ca7453
Content: Insider preview for Microsoft HoloLens
Content Source: hololens/hololens-insider.md
Product: hololens
Technology: windows
GitHub Login: @scooley
Microsoft Alias: scooley

There you go. PR https://github.com/MicrosoftDocs/Hololens/pull/2 is now available(!) for review.

At first, I was surprised that the HoloLens pages don't produce tickets in their own repository, given that the Pull Requests are placed in the HoloLens repository.
Then again, I presume that issues may gain a wider audience when they are posted to the Windows IT-Pro Docs issue ticket repository.


Fix Atom package description

The atom page for this package says "linter-swiftc" in the main heading :)

It still says that in the readme :D https://github.com/AtomLinter/linter-swiftlint#linter-swiftc
cc @k2b6s9j


Custom Functions on Gradients/Gradient Norms

For research purposes, the ability to define custom functions on the gradient as a part of the optimizer would be very useful. It would be in line with the implementations of clipnorm/clipvalue. This would allow different functions other than simply clipping to be experimented with. An example of the utility of this functionality is demonstrated by the creation of the RevReLU layer in https://arxiv.org/pdf/1612.02766.pdf, which uses the ReLU function on the gradients to perform weakly supervised semantic segmentation.

Ok, there's not really a question here. But to add something LIKE clipnorm, you should read the code in the keras/optimizers.py file. For instance, here is the code which supplies the gradient updates for the SGD optimizer:
    def get_updates(self, params, constraints, loss):
        grads = self.get_gradients(loss, params)
        self.updates = []

        lr = self.lr
        if self.initial_decay > 0:
            lr *= (1. / (1. + self.decay * self.iterations))
            self.updates .append(K.update_add(self.iterations, 1))

        # momentum
        shapes = [K.get_variable_shape(p) for p in params]
        moments = [K.zeros(shape) for shape in shapes]
        self.weights = [self.iterations] + moments
        for p, g, m in zip(params, grads, moments):
            v = self.momentum * m - lr * g  # velocity
            self.updates.append(K.update(m, v))

            if self.nesterov:
                new_p = p + self.momentum * v - lr * g
            else:
                new_p = p + v

            # apply constraints
            if p in constraints:
                c = constraints[p]
                new_p = c(new_p)

            self.updates.append(K.update(p, new_p))
        return self.updates

As you can see, the gradients are clearly available during this function call (which happens after each training batch), and thus you can easily make your own custom optimizer and add any such functions as you'd like!

Sorry, should've reframed as a question - shouldn't have tried to write this on a bus on my phone in the morning either because I wrote this question pretty incorrectly. My question is "how would you go about using the gradients for some arbitrary tensor X (for a convolutional neural network) in the network to modify the activations of X between each update"?
I realized it's trivial to apply a function to the gradients, but it's unclear how you would modify the activations using the gradients from the last update before moving forward to the next. Thanks for the concise answer on my poorly-phrased question though. It has made me think about a potential implementation: if I can access the loss between each update, I can get the gradient for the parameters in question, then manually change the activations by the value of the gradient (if the gradient is less than zero, have set the activation to zero, while if the gradient is greater than zero, then keep the activation).

Yes, that is the params argument in the get_updates() function - it is the weights of the network, and are ONLY modified in that get_updates() call. If you want to modify your weights, it should be there or in a custom optimizer in your case.
If you want to modify activations (which are not trainable), you need to use something like the BatchNormalization() layer, but again, custom. How would you access the gradients from such a layer - is that your question? I have a good idea, how about you make a custom optimizer and add a class property of the layer that STORES the gradients! Here's what it might look like:
def get_updates(self, params, constraints, loss):
        grads = self.get_gradients(loss, params)
        self.my_layer.stored_gradients = grads # ADD THIS
        self.updates = []

        lr = self.lr
        if self.initial_decay > 0:
            lr *= (1. / (1. + self.decay * self.iterations))
            self.updates .append(K.update_add(self.iterations, 1))
....

Now, of course you might have to pass in the layer as an argument to your custom optimizer:
myopt = CustomOpt(my_layer=my_layer)

But then when you make that custom layer, you can do as follows:
class CustomLayer(Layer)
     def __init__(self):
          self.stored_gradients = []
... blah ...
    def call(self, activations):
        current_gradients = self.stored_gradients # should be updated from custom optimizer
        # alter the activations however and return them
        new_activations = activations * K.mean(current_gradients)
       return new_activations

Overall, your model might look like this:
input = Input(shape=..)
dense =  Dense(100,activation='relu')(input)
custom = CustomLayer(...)(dense) # this layer will take dense activations and do something

Thank you! That will work perfectly for my purposes - I'm just delving into writing custom layers for the first time (never needed to before) and it's a bit of a daunting task at first. I'll leave this open for a second while I get the custom layer configured - once I'm done I'll post it here if you find that useful (the RevReLU layer is an interesting idea and definitely might be interesting to others working with weakly labeled data).

@ncullen93 I want to apple PSO optimizer to my network within keras. how can i modify Keras.optimizer.py? Because there is no need of gradient in my PSO code.

@hiteshnitetc were you able use PSO?

yes
On Fri, Nov 10, 2017 at 10:47 AM, Bhaskar Dhariyal<EMAIL_ADDRESS>
wrote:

@hiteshnitetc https://github.com/hiteshnitetc were you able use PSO?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/5142#issuecomment-343377231,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AdMpSjWx9hXbaVOA7gG3AWjUcSg5brjoks5s09xvgaJpZM4LraZq
.

@hiteshnitetc Can I have your mail address?

<EMAIL_ADDRESS>On Sun, Nov 12, 2017 at 8:11 AM, Bhaskar Dhariyal<EMAIL_ADDRESS>wrote:

@hiteshnitetc https://github.com/hiteshnitetc Can I have your mail
address?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/5142#issuecomment-343709366,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AdMpSueIBDc3HwBjX3AUGPKAX9QnpmYnks5s1lrbgaJpZM4LraZq
.

@hiteshnitetc you got PSO to work with Keras? Any code if so? Would be interesting to see.

It's not completely on keras , it's uses some functions of keras like convolution and losses. I have write weight updation by using numpy, because i have not much knowledge about keras ops


Typescript support

Manifest and maybe other cli should support .ts and .tsx files
I think the transform will just work as babel 7 supports typescript

collectFBT also throws when parsing typescript files with babel
Error: [BABEL] unknown: Configuration contains string/RegExp pattern, but no filename was passed to Babel
/entria-fullstack/node_modules/babel-plugin-fbt/bin/collectFBT.js:131
    throw new Error(`Failed in ${errCount} files`);
    ^

Error: Failed in 1 files
    at writeOutput (/entria-fullstack/node_modules/babel-plugin-fbt/bin/collectFBT.js:131:11)
    at ReadStream.<anonymous> (/entria-fullstack/node_modules/babel-plugin-fbt/bin/collectFBT.js:169:5)

after https://github.com/facebookincubator/fbt/pull/51
I've got this error:
Error: Line 37 Column 11: `fbt` is not bound. Did you forget to require('fbt')?
/entria-fullstack/node_modules/babel-plugin-fbt/bin/collectFBT.js:131
    throw new Error(`Failed in ${errCount} files`);
    ^

Error: Failed in 1 files
    at writeOutput (/entria-fullstack/node_modules/babel-plugin-fbt/bin/collectFBT.js:131:11)
    at ReadStream.<anonymous> (/entria-fullstack/node_modules/babel-plugin-fbt/bin/collectFBT.js:169:5)
    at ReadStream.emit (events.js:187:15)
    at endReadableNT (_stream_readable.js:1094:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

it could be related to https://github.com/facebookincubator/fbt/issues/46

this could help https://github.com/sibelius/codemod-RN24-to-RN25/blob/master/transform.js#L22

Support for Typescript files is certainly on our radar but not scheduled yet.
Re: the error about the absence of require('fbt'). That check was useful before when we had no proper way to check that fbt was a defined variable in the JS module scope. Now that we've got better Flow support, we can remove this.

@kayhadrin Do you have any pointers where that check is made? I wouldn't mind keeping a fork going until things are ready for typescript support.

@baluubas you can use my 2 pull requests
https://github.com/facebookincubator/fbt/pull/50
https://github.com/facebookincubator/fbt/pull/51

@baluubas I believe @sibelius answered your question.

@sibelius Re: https://github.com/facebookincubator/fbt/issues/49#issuecomment-462069077
Could you provide an example of how you're importing fbt in your JS module?
I assume it's something like this:
import fbt from 'fbt';
// or
import {fbt} from 'fbt';

I’m using export default
Checking the code, I could use named or export default
For me, anyone of them could work

I've applied the your two PRs locally (#50 and #51),
converted the demo-app/src/example/Example.react.js file to tsx,
ran yarn manifest and yarn collect-fbts and it worked successfully.
If that's still an issue on your side, you may want to submit the minimum repro code in a Codepen.

It seems the pull requests were merged, can this issue be closed or is Typescript still not fully supported?

I'll close for now, and if any further issues arise, we can investigate.

Here is an article about  Typescript + fbt where I describe problems that i faced and how to fix them


F# Support

The Xamarin Inspector team informs me that the F# compiler doesn't currently work on iOS.
To make F# work we need to send newly compiled assemblies instead of sending code to the device.
I haven't experimented at all with this and just hope we can make it work. :-)

I actually took a stab at assembly-based 'live coding' some months back. It was kind of like a poor-man's Continuous. I've unhidden it here and there might be some code and/or learnings that can be taken from it. It was (again) designed to be too general; so excuse any odd looking parts.
The core part on the device was InjectorHostBase (code), roughly analogous to your http server. It was socket based and handled receiving messages including new versions of assemblies. You would then subclass InjectorHostBase to create different 'plugins'. For example, this CocosInjectorHost (code), which was a 'plugin' for CocosSharp. The core for the plugin was to override ShouldLoadAssembly to decide whether or not to look at a new assembly, and then ProcessNewAssembly which would do the actual work. In CocosSharp case, (broadly) it would look at the incoming assembly for classes decorated with the Inject attribute. If the class didn't exist in the current scene, it would create it and add it. If the class did exist, it would swap out the old implementation with the new one by removing the old node and creating the new version, putting it in the previous node's position. On the client (computer) side, the current method was to use a FileSystemWatcher(code) to see new assemblies and send them off to the injector host. You had to open a second instance of visual studio and ctrl-shift-b to build; so it was certainly not the nice experience you get with Continuous. I did also experiment with live-watch variables, where you could decorate a property on an object and provide a time interval at which the value would be sent back to the client.
This worked fine on desktop and android. On iOS, I first hit an issue where the simulator would not load an updated implementation of an assembly. It could load a new assembly, but when you got a second implementation of it, the load would appear to succeed, but the old implementation would remain. I got around this by using Mono.Cecil to rename the incoming assembly. But then the type registrar would complain about the types in the new assembly already being registered. This is the point that I kind of lost steam at, but I think it would probably also have been solved by renaming the types (I guess that's why in Continuous, reloaded instances have a long suffix of characters against the name?).
There are some advantages to assembly-based reloading. For example, if you reference any new libraries, they can be sent to the running application along with your own updated assembly. Still, I wonder if you could still get the tight integration you have with Continuous now and how difficult it would be to do things like your most recent feature - the inline REPL evaluation.
Anyway, hopefully some of this is interesting/of use.


[Idea] New controller layout for MCXR!!

+1

Even if the mod author doesn't change the controls I think this is still a great visual layout to have on the Curseforge mod page.


Coin amounts mismatch

System: sm-mini
App version: 0.0.6

Behavior: Balance: 446.602 smh but yearly coin 276.126 SMH. Smeshed: 393.314 SMH. Send: 5.0.2 SMH.
Expected: yearly coin stats to correspond to true balance.

Hi. Why was this closed without any code changes?

@lrettig it was fixed while fixing another issue

@IlyaVi gotcha - can we link that issue here? Thanks!

actually it is linked to PR #492
see it above

@IlyaVi thanks


Support Firebird RDBMS

Firebird is an excellent high performance database that is really easy to install and works right out of the box. We use it in a number of important projects via the node-firebird package but we'd like to move to a common PRISMA ORM model.

Hi there,
As part of our effort to clean up our GitHub repo, we’re closing this issue to help focus on features and improvements that align with our focus areas and those of our community.
If this is issue is still important to you, feel free to reopen the issue and get like-minded users to upvote as we will prioritize fixing issues based on community interest. If you do reopen the issue, please ensure that there’s a clear method for us to reproduce it.
For more insight into our priorities and how we’re thinking about the future of the Prisma ORM, please see our recent blog post: https://www.prisma.io/blog/prisma-orm-manifesto.
Thank you for your understanding and support!