Which shard is at which node? It seems like all shards are on the same node.

I installed qdrant helm chart in cluster with 1 node master, 4 node workers and I created a collection with shard_number=2, replication_factor=2.
When I get cluster info with command:
curl http://<IP_ADDRESS>:31990/collections/test_collection_1/cluster # 6333:31990
The results is:
{
   "result":{
      "peer_id":7901933018698687,
      "shard_count":2,
      "local_shards":[
         {
            "shard_id":0,
            "points_count":52505,
            "state":"Active"
         },
         {
            "shard_id":1,
            "points_count":47495,
            "state":"Active"
         }
      ],
      "remote_shards":[

      ],
      "shard_transfers":[

      ]
   },
   "status":"ok",
   "time":0.000017769
}

It's just get info about the number of shards, number of points in each shard, but I want to know which shard is on which node. It seems like all shards are on the same node. And I guess it's because the memory in other nodes not increase and I haven't run 2 command lines:

./qdrant --uri 'http://qdrant_node_1:6335/'
./qdrant --bootstrap 'http://qdrant_node_1:6335/'

Are these commands run in the container? But how to access container in kubernetes (I tried "kubectl exec -it my-qdrant-demo sh", etc, but error: unable to upgrade connection: pod does not exist?)
If not, where are these commands run after install qdrant helm chart?

Why there is no any remote shard (backup)? Is it relate to pv/pvc ? (I'm using pv with hostPath /home/data in each node)

If your qdrant pod isn't in the default namespace of your cluster, that command won't be able to find the pod. Specify the namespace in the command "kubectl -n <namespace> exec -it my-qdrant-demo sh"

Hi @ngominhhaibk , those ./qdrant commands are not something you need to worry about since the helm chart configures those commands automatically when you increase the replicaCount.
As for the shards, by default all shards will stay on the original node (since "remote_shards" is empty, that means all your shards are on one node). Qdrant does not automatically move shards around between your nodes. Here are the docs about moving shards: https://qdrant.tech/documentation/guides/distributed_deployment/#making-use-of-a-new-distributed-qdrant-cluster

Thank for all your helps. I resolved these problems.


avoid using unstable ptr::invalid_mut

I want to rename that function in https://github.com/rust-lang/rust/pull/117658, but currently the build fails because hashbrown uses it.

@bors r+

:pushpin: Commit f13a05ab71afdb3b6389e8ba1b8204e475fc66fb has been approved by Amanieu
It is now in the queue for this repository.

:hourglass: Testing commit f13a05ab71afdb3b6389e8ba1b8204e475fc66fb with merge 2e138e9481033b7eceae9b6f4fa2fe42c4f88b45...

:sunny: Test successful - checks-actions
Approved by: Amanieu
Pushing 2e138e9481033b7eceae9b6f4fa2fe42c4f88b45 to master...


commentOnId uses objectId

I realized that commentOnId uses objectId which must be obtained from the metadata, users have to first use get_documents in order to obtain the objectId for a document. Alternatively, documentIds are public-facing on regulations.gov and included in the metadata.
Not sure if one obtains more information than the other but flagging this as something to deal with in documentation and maybe deal with later. I don't want to create two functions that do the same thing ...

yes, our current draft documentation specifies two functions:
Get metadata for comments on a document or docket

get_comments_on_docket("[docket_id]") retrieves all comments for a docket (e.g., including an Advanced Notice of Proposed Rulemaking and all draft proposed rules)

get_comments_on_document("[document_id]") retrieves comments on a specific document (e.g., a specific proposed rule)

However, they could probably just be one get_comments function that takes a docket =  or document_id argument...or maybe even detects whether a docket id, document id, or object id is supplied.

I updated the make_path for CommentId to detect what type of ID someone is feeding it, but it currently only works for objectIds because it seems as though you can't tag on filters by lastmodifieddate, sort, or anything of that sort when using the documentId.
On the API help page they have an example using "FDA-2009-N-0501-0012" - the API returns results if you just use their simple call with either include attachments or without it, but you can't do much beyond that simple call.


are git hooks not triggered?

hi,
I was wondering if – when using nodegit – any present git hooks are triggered?
it doesn't seem so. post-commit hooks for instane, only run when I commit manually from the command line.

Hey @freder so libgit2 (the library which we're wrapping) does not (and apparently will not) support hooks (see here). It does look like they want to put in a layer that allows applications to easily call them but that's not currently in.

@johnhaley81 It could be something nodegit could support itself, it makes sense it's not part of libgit2. See https://github.com/libgit2/libgit2/issues/964#issuecomment-170353017


latest

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 11 committers have signed the CLA.:x: hileomsi:x: ntegral:x: reastyn:x: swift-nicholas:x: hardydex:x: weslleybraga:x: juzerali:x: dependabot[bot]:x: uptownhr:x: holm:x: xiifainweslleybraga seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


rkt: make image list order flag case-insensitive

We should treat options as case-insensitive

I am not really sure I buy it, what's the motivation? Most CLIs are case-sensitive, no?

I don't think there is a clear standard way of doing it. For example if we look at the --color flag of bot ls and grep we'll find that grep  is case-insensitive while less is more strict. Generally, I'd say if the option is internal to the command we should treat the option as case-insensitive; basically where intent is clear and it's innocuous.
The main motivation here, however, was to make the treatment of the options consistent within this command itself. If we look at the code for parsing the --fields flag, we'll find that it's treating the options as case-insensitive.

I'm sold, lgtm


Fix comments that were ported from YAML to FTL

Some YAML comments were accidentally ported to Fluent files by #7916.  Since Fluent doesn't have inline comments, they show up in the game.

fastest merg in the west...


Problem with HPC pro and Falcon

Hi,
I have got the following error with  install_unzip_180312.sh:
[5711]$(u'lfs setstripe -c 12 /lustre/scratch/waterhouse_team/benth/falcon')
[5711]Call u'lfs setstripe -c 12 /lustre/scratch/waterhouse_team/benth/falcon' returned 0.
[INFO]Setup logging from file "None".
[INFO]fc_run started with configuration fc_run.cfg
[ERROR]Failed to parse config "fc_run.cfg".
Traceback (most recent call last):
  File "/lustre/work-lustre/waterhouse_team/apps/falcon_unzip/fc_env_180905/lib/python2.7/site-packages/falcon_kit/mains/run1.py", line 136, in main1
    support.parse_config(input_config_fn))
  File "/lustre/work-lustre/waterhouse_team/apps/falcon_unzip/fc_env_180905/lib/python2.7/site-packages/falcon_kit/run_support.py", line 167, in get_dict_from_old_falcon_cfg
    sge_option = config.get(section, 'sge_option_da')
  File "/work/waterhouse_team/miniconda2/envs/falcon_unzip/lib/python2.7/ConfigParser.py", line 618, in get
    raise NoOptionError(option, section)
NoOptionError: No option u'sge_option_da' in section: u'General'
Traceback (most recent call last):
  File "/lustre/work-lustre/waterhouse_team/apps/falcon_unzip/fc_env_180905/bin/fc_run", line 11, in <module>
    load_entry_point('falcon-kit==1.0+git.6bb3daa96931aece9bd3742bccc77ad257b7bb65', 'console_scripts', 'fc_run')()
  File "/lustre/work-lustre/waterhouse_team/apps/falcon_unzip/fc_env_180905/lib/python2.7/site-packages/falcon_kit/mains/run1.py", line 644, in main
    main1(argv[0], args.config, args.logger)
  File "/lustre/work-lustre/waterhouse_team/apps/falcon_unzip/fc_env_180905/lib/python2.7/site-packages/falcon_kit/mains/run1.py", line 136, in main1
    support.parse_config(input_config_fn))
  File "/lustre/work-lustre/waterhouse_team/apps/falcon_unzip/fc_env_180905/lib/python2.7/site-packages/falcon_kit/run_support.py", line 167, in get_dict_from_old_falcon_cfg
    sge_option = config.get(section, 'sge_option_da')
  File "/work/waterhouse_team/miniconda2/envs/falcon_unzip/lib/python2.7/ConfigParser.py", line 618, in get
    raise NoOptionError(option, section)
ConfigParser.NoOptionError: No option u'sge_option_da' in section: u'General'

with the following cfg file:
[General]
input_fofn = ../fasta2DB_input.fofn
input_type = raw

seed_coverage = 40 #(for Banana Diploid) and 40 (for Nbenth)
genome_size = 320000000 #(for Banana Diploid) and 320000000 (for Nbenth)

length_cutoff = -1
length_cutoff_pr = -1

job_type = string
pwatcher_type = blocking
job_queue = qsub -S /bin/bash -V -q lyra -N ${JOB_ID} -o "${STDOUT_FILE}" -e "${STDERR_FILE}" -l nodes=1:ppn=${NPROC},mem=25gb -W umask=0007,block=true "${CMD}"

pa_concurrent_jobs = 96
cns_concurrent_jobs = 96
ovlp_concurrent_jobs = 96

pa_HPCdaligner_option =  -v -B128 -M32 -e.70 -l4800 -s100 -k18 -h480 -w8
ovlp_HPCdaligner_option = -v -B128 -M32 -h1024 -e.96 -l2400 -s100 -k18

pa_DBsplit_option = -a -x500 -s200
ovlp_DBsplit_option = -s400

falcon_sense_option = --output_multi --min_idt 0.70 --min_cov 2 --max_n_read 100 --n_core 8
falcon_sense_skip_contained = True

overlap_filtering_setting = --max_diff 80 --max_cov 80 --min_cov 2 --n_core 12

How would I convert sge to HPC pro?
Thank you in advance.
Michal

Is this bioconda recipe the same to your recommended version?
Thank you in advance,
Michal

pb-falcon has the Python code, pb-assembly pulls that along with all dependencies.


refactor(docs): Modularize docs

Adds modules to the docs - attempt to make them more accessible (see #1506)
Updates the README to point to the new docs

Closes #1498.

well I need to redo this 🤦♂


Support for VisualWebArena evaluation in OpenHands

End-user friendly description of the problem this fixes or functionality that this introduces
Implementation of VisualWebArena agent in OpenHands for benchmarking results.

[ ] Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions
Replicate the implementation of VisualWebArena agent using OpenHands and BrowserGym frameworks.

Link of any specific issues this addresses

cc @ryanhoangt who may also be interested in taking a look

It would be nice to add an integration test here https://github.com/All-Hands-AI/OpenHands/tree/main/evaluation/integration_tests
These scripts are now eval scripts, running a simple task, just to see that the whole is running and finishing.

@adityasoni9998 just a gentle ping to check if this is still on your radar.

@adityasoni9998 apologies for the double ping, just want to check if this is still in progress?

@adityasoni9998 apologies for the double ping, just want to check if this is still in progress?
Hi @mamoodi, sorry for the late response. I have been a bit busy for the last couple days. Will try to complete it this week.

@adityasoni9998 apologies for the double ping, just want to check if this is still in progress?

Hi @mamoodi, sorry for the late response. I have been a bit busy for the last couple of days. Will try to complete it this week.

No worries! Just checking that someone is still working on this. Thanks for your response!


cat pack does not decrypt pack

Output of restic version
restic 0.3.3 (v0.3.3-0-g4d93da9)
compiled with go1.7.4 on linux/386
Expected behavior
restic cat pack returns decrypted pack file
Actual behavior
restic cat pack returns the pack as stored on disk.

Thanks for raising this issue. I'm afraid that you understood something wrong: A pack file is a collection of encrypted blobs, followed by a header, followed by the header length. The definition can be found in the file Design.md. There is no "decrypted pack file".
So, I'm going to close this issue since restic cat pack should indeed only write a pack file to stdout, without decrypting anything.
If you need any further explanation or have an idea on how to improve the documentation, please leave a comment.

There isn't really any use for decrypting the pack file in day to day operations, but I was curious of the internals. I'm new to Go, so I made a quick tool in Python for inspecting/decrypting the pack files. (https://github.com/oysols/restic-python) I agree with you that there really isn't any good reason to include this in restic.
Anyways; Keep up the good job. :)


Shutdown all but one CPU cores on screen off

Overview
This PR attempts to reduce power consumption by powering down all but one CPU cores while the device remains awake, yet inactive. Then, once the user interacts with the device again, all CPU cores are powered back on.
Notes
While this should reduce power consumption to some degree, I am unsure of how much actual power is saved on the PinePhone. Some quick experiments did not show a noticeable improvement, but YMMV.

Sorry, have been quite busy in the past weeks.
Do you still use this script? If so, how is your experience regarding the power savings? Does everything else behave normally (e.g. receiving a call)?
Will test it in every day use in the next couple of days.
Thank you.

Do you still use this script?

I am using a slightly modified version of this script, with fine-tuned values for SLEEP_SECS_INIT, SLEEP_STEP, SLEEP_MAX_SECS and WAKE_SECS, according to own experience. Also, I have disabled some features that were not so interesting for me, such as notification LED or single CPU core use (see notes below).

If so, how is your experience regarding the power savings?

I am using the modified version above everyday, and is a good compromise between convenience and battery life. Typically, battery would last around 12-20 hours with light-medium usage, which is a lot more than compared to no suspend (and runs noticeably cooler, too), although less compared to keeping the phone suspended until the power button is pressed.
Surprisingly, the power saving feature suggested on this PR was not very useful to me, due to the following reasons:

Consumed power and heat dissipation was similar (if not worse) with 1 CPU core. I have used the following script to print real-time power usage over stdout.
The phone would take several seconds to wake up from suspend when running on a single CPU core, which was a bit inconvenient, compared to its increased responsiveness with all CPU cores active.

Does everything else behave normally (e.g. receiving a call)?

Calls and SMS are working fine so far. However, the Quectel EG25 modem on the PinePhone is known to be buggy, so the modem might crash more often, compared to no suspend. According to own experience, surprisingly modem crashes seem to occur more often when on the go.
As a workaround, I have written the following script, which restarts eg25-manager if the modem crashes. I did not work so well for Manjaro + Phosh, but works like a charm on Arch Linux + Phosh, so I do not have to worry about modem crashes any more.
OTOH, I am using some applications, such as Dino, which take a significant amount of time to receive incoming messages for some reason unrelated to sleepwalk, so I had to assign WAKE_SECS to a value higher than I would like to. Hopefully, these issues are solved anytime soon so as to keep WAKE_SECS as low as possible and thus increase battery life.
Sometimes though, the phone crashes when entering suspend and thus is unable to wake up, even if the power button is pressed. Typically, the phone would sleep for a specific amount of time and then wake up, as shown below:
Apr 08 15:42:26 danctnix sleepwalk[2955]: Sleeping for 90s
Apr 08 15:42:26 danctnix kernel: PM: suspend entry (deep)
Apr 08 15:43:57 danctnix kernel: Filesystems sync: 0.054 seconds
Apr 08 15:43:57 danctnix kernel: Freezing user space processes ... (elapsed 0.018 seconds) done.

However, sometimes it cannot wake up and thus must be force-restarted. For example, as shown below, the phone crashed at Apr 08 14:48:37 and did not report anything else on journalctl until rebooted.
Apr 08 14:48:37 danctnix sleepwalk[2977]: Sleeping for 90s
Apr 08 14:48:37 danctnix kernel: PM: suspend entry (deep)
-- Boot<PHONE_NUMBER>c54b2d83e3c0b14e715531 --
Apr 08 15:37:09 danctnix kernel: Booting Linux on physical CPU 0x0000000000 [0x410fd034]

In any case, I assume this issue is not related to sleepwalk, but possibly crust.

Thanks for your answer.
I have mostly the same issues as you. Unfortunately we have to deal with this modem on the pinephone pro, too (currently collecting dust in my closet). But I'm hoping that the open source driver will mature in the future. I will try your modem-restart script and see how it works for me. I'm using arch linux arm, too.
The 30 seconds wake time is ok for me (using mainly telegram-desktop for messaging). Usually I get my messages within 10-15 seconds I think. Since the script does now detect receiving notifications we could implement some sort of smart waiting depending on the wait time of previous cycles someday.
A small note to your modified version of the script: You seem to check /sys/class/power_supply/axp20x-battery/status to detect, if the pinephone is charging or not. This will however report Discharging when the pinephone is charged 100%, but still connected to a charger. The solution is to check /sys/class/power_supply/axp20x-usb/online instead (1, when charger is plugged in, 0, when not).

Closing this pull request, because it isn't useful.

A small note to your modified version of the script: You seem to check /sys/class/power_supply/axp20x-battery/status to detect, if the pinephone is charging or not. This will however report Discharging when the pinephone is charged 100%, but still connected to a charger. The solution is to check /sys/class/power_supply/axp20x-usb/online instead (1, when charger is plugged in, 0, when not).

Good point! I had just realized about this behaviour.


[Path] Add Dependency "farming"

I wanted to suggest the following patch with creates "depends.txt" and adds "farming" as a dependency since spiders drop "farming:string".
spidermob-depends.txt.diff.zip

You're right, thank you for your suggestion :)
https://github.com/Darcidride/minetest-spidermob-v1/commit/c72d2ff3091c231a15692d774b04764328892190
But be carful, this mod isn't update since a long time with the mobs redo API, i works with Minetest 0.4.14 easily, but can works with better performances if extracted from mobs redo.


Support setting persistent volume reclaim policy

Is your feature request related to a problem? Please describe.
persistentVolumeReclaimPolicy isn't an option under PersistentVolumeClaimSpec v1 core I would like to be able to set it to "Delete" for our solrcloud test pipelines where I don't want the PVCs sticking around after deletion.
What would it take to be able set different values for persistentVolumeReclaimPolicy?
Do we need orphan pvc cleanup code logic like I see in zookeeper-operator? Or will just setting this do the cleanup for us?
Describe the solution you'd like
Ability to configure under dataPvcSpec "Delete" for persistentVolumeReclaimPolicy
Describe alternatives you've considered
In the meantime ill write a cleanup script outside of operator in our pipeline.

@sepulworld This is definitely a feature we need to add.
Unless I'm misunderstanding, I am pretty sure that the reclaimPolicy is a feature of the PersistentVolume, not the PVC. It says if the PV is unmapped from the PVC, what should happen to it. The only way that the solr operator could use this is for users to specify that they want a certain reclaim policy for their PVs by specifying it in the storageClass parameter of their pvcSpec.
What we need to do is delete PVCs after the statefulSet has been deleted. This would require using the finalizers much like the Zookeeper Operator does. I think following their lead would be good here. I think this is also an opportunity to refactor how we want the spec to look for this. Since we will have to create a new persistence option (or another name) that the pvcSpec and deletionPolicy will live under. We can make this backwards compatible at least for a minor version. But we will have to move things around eventually.

The finalizer makes sense!

I think we need an overhaul of the storage settings.
Something that looks like:
Spec:
    storage:
        storageType: <ephemeral,persistent>
        persistence:
            pvcSpec: <PVCSpec object>
            reclaimPolicy: <Reclaim,Delete>
         ephemeral:
            emptyDirVolumeSource: <EmptyDirVolumeSource object>

This is very similar (if not exactly the same as) the options soon to be in the Zookeeper Operator. Maybe we rename to dataStorage just to make sure that it's separate from the backupRestoreVolume stuff. Or maybe that moves into this section? Definitely some options to think about.

@swarupdonepudi was looking into this. We have to manually or script the cleanup of PVCs presently. Would be sweet to build in some of this automation into solr-operator. Is there an open PR with zk-operator for something new?

There is a PR for adding ephemeral storage and cleaning up the section in their CRD https://github.com/pravega/zookeeper-operator/pull/215. But the other functionality is already there.


[rnet] Simple and easy-to-use network module for raylib

As already mentioned several times by raylib community, a network module is a missing piece on raylib. Long ago I started drafting a possible network module API for raylib but due to lack of resources, project was stopped... I start this issue to list project specifications and references.
raynet is a simple and easy-to-use network library. Some desired features:

Available as raylib module (raynet.c) but also usable as a  standalone library
No external dependencies (only the standard system libraries - winsock2.h/socket.h)
Self-contained in a single file (header-only) or two portable files: raynet.h + raynet.c
Focused on UDP protocol (probably the most useful for games)
Support WEB (WebSockets) and ANDROID platforms (if possible)

Just as a quick reference, the following APIs have been analyzed:

Unity network-transport layer - Really good API
SDL_Net - Only UDP pieces
SFML Sockets - Not bad...
Single file network libraries - Special mention for sts_net

I'm not an expert on network programming, actually last time I remember doing some sockets code was on University... but here it is basic raynet API proposal (probably very improvable):
void InitNetworkDevice(NetworkConfig config);
void CloseNetworkDevice();
int NetworkConnect(int hostId, const char *address, int port);
void NetworkDisconnect(int hostId, int connectionId);
void NetworkSend(int hostId, int connectionId, int channelId, void *data, int length);
int NetworkReceive(int hostId, int connectionId, int channelId, void *data, int length);
// Maybe hostId, connectionId and channelId can be a `struct NetworkSocket`.

Just keep in mind this API is a very rough proposal...
EDIT: Added to list desired support for WEB and ANDROID.

NetworkDisconnect and NetworkSend could both fail, they should return an error code or give the user some way to find out what went wrong (e.g. Network is down or a timeout occurred). Speaking of timeout, how will NetworkReceive work?
Blocking till a packet arrives doesn't sound too useful, so I assume it will poll for new packets and return immediately?

Probably shouldn't use something so low level as raw tcp/udp style interfaces.  Should probably build raynet around something like https://github.com/SLikeSoft/SLikeNet or so instead as they have actually useful constructs for games.

I think a NetworkSocket and NetworkData structs would be a good way to do it, with the latter having a void pointer that the user can set as their data.

One haircut later.
i have some input.
// This call can fail on windows, it is rare but should be accounted for.
bool InitNetworkDevice(NetworkConfig config);

//Why is the host included here?
bool NetworkDisconnect(int hostId, int connectionId);

//This should return something, maybe not int but atleast a bool.
//But giving the user an option to track the sent bytes is useful
//Why is hostid included here?
//What is channelId supposed to be used for?
int NetworkSend(int hostId, int connectionId, int channelId, void *data, int length);

//Why is hostid included here?
//What is channelId supposed to be used for?
int NetworkReceive(int hostId, int connectionId, int channelId, void *data, int length);

I'll make this a separate thing, focusing on UDP and implementing WebSockets seems sort of weird?
I might have a different notion of what a WebSocket is in that case please explain.
Provided that the idea is to be able to make a GET and POST request this should not be impossible, however then we should sit down and focus on what part of the HTTP protocol, we would want since posts and gets can be very funcy if you let them.

Probably shouldn't use something so low level as raw tcp/udp style interfaces. Should probably build raynet around something like https://github.com/SLikeSoft/SLikeNet or so instead as they have actually useful constructs for games.

Raylib is written in c99 so this library could not be used if we want to keep it that way.
See bullet point 2 in rays post.

Raylib is written in c99 so this library could not be used if we want to keep it that way.

It itself is yes, hence why it should be entirely wrapped, it should not be exposed to the user of the library at all.

No external dependencies (only the standard system libraries - winsock2.h/socket.h)

It's license allows baking it into a project so it's no longer an external dependency.

Thanks to all of you for your answers! :)
@a3f @Skabunkel yes, functions should return some state info.
@a3f don't know yet how those functions should properly work.
@Gamerfiend agree.
@Skabunkel hostId is probably not required, I took it from Unity network API. Some basic encoding function could also be useful to encode/decode packages. About WebSockets, I understand is the equivalent of sockets to run on web, maybe I'm wrong with my assumption.
@OvermindDL1 Thanks for the sugestion but that's a big library, I prefer to keep it simple and self-contained.

Ah, that makes sense. unsure how that would work in webasm but the joy of exploration.

Android NDK sockets
https://github.com/wzbos/Android-NDK-Socket/tree/master/sample/app/src/main/jni
tcomm.c

Why so low level? Some things like packet sequencing and relability would be nice (something like eg. ENet, look at packet flags), those things are hard to make for new programmers and would help them with networking.

@zaknet I've been working on small network module for raylib - hence why this issue was opened, I plan to have a similar api to sfml/sdl_net for those who want to use platform agnostic socket calls.
I've also got plans to include a, as you put it 'higher level' game api on top, the goal is to support basic replication, serialisation, callbacks, sequencing etc for those less familiar with networking

With raylib having quite a minimalist api however, my roadmap for the module is very much subject to change. Within the week I'll likely post a spec, outline what features I intend to add and we can have a discussion about what belongs in an api call, and what we can provide as examples :)

@Skabunkel Thank you very much for the links! I looks pretty good! Supporting WEB and ANDROID has been added to the list!
@zakrent I agree, intention is to offer a simple and easy-to-use API but I'd start with a bare-bones working solid version first.
@Syphonx I'm following your rnet implementation, you're doing a great work! I think you're using TCP at this moment, are you planning UDP support also? We can discuss it later on once specs published... Keep up the good work! :)

By the way, @Syphonx, do you think we could have a functional version to be included on raylib 2.5 in the following two months or so?

I plan to have barebones support for TCP/UDP, a low(er)-level socket api, serialisation helpers and some examples done in the next week or so @raysan5

Features like replication, data packet tags and more advanced features will likely be added on an 'as we go' basis

An attempt to use any networking library that includes winsock2.h on Windows, will result in name colliding and conflicts with Raylib functions and structures at the compilation, specifically: Rectangle, UnhideWindow, CloseWindow, ShowCursor, and CloseWindow.
Windows headers already have these declarations. I'm surprised that you don't follow an appropriate naming conventions with a vendor-specific prefix, this is quite trivial thing in C.

I used Raylib in NetDynamics if you're interested. I've added Ray prefix to all functions and renamed Rectangle to Rectangle2D. Without these modifications, it's impossible to build Raylib on Windows with included winsock2.h.

@nxrighthere Hey! Excuse me for this super-late response, I completely forget about this issue...
About windows.h inclusion (or related headers), symbol collisions could be mostly avoided like explained on #857, also embedding the inclusion into its own module.
Personally I don't like prefixes (when I started raylib I came from C# world) but I agree it's the best option in C to avoid that kind of collisions... maybe in a future update of raylib...
Congrats for the NetDynamics, very nice project! Great code organization and github project presentation. I like it!

@raysan5 Thanks, I glad that you like it.

Personally I don't like prefixes (when I started raylib I came from C# world) but I agree it's the best option in C to avoid that kind of collisions... maybe in a future update of raylib...

Well, it's not a problem for me I've just made a small script that renames everything. But, for example, people to whom I suggested Raylib in my community encountered the same problem, and they were really unhappy with that, so I think it would be better to solve this out of the box in the traditional approach.

Just for reference, I found swrap library, looks very interesting!

Such stuff is just a very basic UDP API, for multiplayer games, you will need a networking library with a mature protocol that carries at least these features:

Secure connection association and management (through cryptography preferable: https://github.com/jedisct1/libsodium or https://github.com/jedisct1/libhydrogen for example)
Efficient reliability to guarantee delivery of data (see https://github.com/skywind3000/kcp for example)
Flow and Congestion Control (should be pluggable and optional, not every project will benefit from it)
Packet fragmentation and reassembly, IP fragmentation should be avoided in any case
Packet multiplexing to efficiently aggregate data when possible

This is only the tip of the iceberg, consider something that provides such feature-set out of the box. Godot is using ENet which is used in many modern games today such as League of Legends for example (they are using cryptography for payload on top).

For inspiration:
https://github.com/networkprotocol/yojimbo

Hey guys! Is someone working on this?

Hi @paezao! Not really, I did some review a while ago but it's not functional at this moment...
My plan was simplifying the API and use UDP-only.

Hey @raysan5, I asked you recently about enet.
But do you actually like enet? I mean, if someone took the source code and put it all together in a single organized file+header, would you use it or does enet lack features that you want?

A single-header fork is available here. It's used in several games in production.

@nxrighthere Oh! Thank you very much for sharing!

What a coincidence @nxrighthere , I recently starred your fork before posting here, I hope @raysan5 likes it. I want to use it for some Unity projects.

Another great alternative: enet single-header fork: https://github.com/zpl-c/enet

zpl-c/enet is the particular repo that I used in my multiplayer pong clone I wrote a couple of days ago. I have little to no background in C and managed to get it up and going fairly quickly which suggests it must be fairly easy to use.

Client snippets: https://pastebin.com/6Y84AzRd
Server snippet: https://pastebin.com/XBwXGeZX

Rough mapping of code snippets to proposed API below:
- InitNetworkDevice()/CloseNetworkDevice() -> Handled by ENet in GameServerConnect()
- NetworkConnect() -> GameServerConnect()
- NetworkDisconnect() -> GameServerDisconnect()
- NetworkSend() -> GameServerPublish()
- NetworkReceive() -> GameServerPoll()

So a thin wrapper over ENet within the same single file seems viable, so long as your happy with the ENet license and whatever maintenance goes along with it. The only other convenience function that would be nice is some sort of transport-layer compression and encryption which should be possible before crafting the ENet packet.
I started having a look at the existing rnet code and for a novice like me it looks like it would be a little easier to maintain/debug than ENet since it is much simpler. That said there would probably be more work in completing it than an ENet wrapper. Effort/maintenance/functionality tradeoff I guess.

zpl-c/enet is the particular repo that I used in my multiplayer pong clone I wrote a couple of days ago. I have little to no background in C and managed to get it up and going fairly quickly which suggests it must be fairly easy to use.

Client snippets: https://pastebin.com/6Y84AzRd
Server snippet: https://pastebin.com/XBwXGeZX

Rough mapping of code snippets to proposed API below:
- InitNetworkDevice()/CloseNetworkDevice() -> Handled by ENet in GameServerConnect()
- NetworkConnect() -> GameServerConnect()
- NetworkDisconnect() -> GameServerDisconnect()
- NetworkSend() -> GameServerPublish()
- NetworkReceive() -> GameServerPoll()

So a thin wrapper over ENet within the same single file seems viable, so long as your happy with the ENet license and whatever maintenance goes along with it. The only other convenience function that would be nice is some sort of transport-layer compression and encryption which should be possible before crafting the ENet packet.
I started having a look at the existing rnet code and for a novice like me it looks like it would be a little easier to maintain/debug than ENet since it is much simpler. That said there would probably be more work in completing it than an ENet wrapper. Effort/maintenance/functionality tradeoff I guess.

@bloodhaze13 thanks for the info! Current rnet implementation was contributed by an user following a proposed API but development was stopped, I tried to pick it up but it required more time than expected, I'm not an expert on networking...
Lately I also keep an eye to another very promising library: https://github.com/nathhB/nbnet

@bloodhaze13 thanks for the info! Current rnet implementation was contributed by an user following a proposed API but development was stopped, I tried to pick it up but it required more time than expected, I'm not an expert on networking...
Lately I also keep an eye to another very promising library: https://github.com/nathhB/nbnet

Hello,
for an old project of mine I used this "Tiny Websockets" library https://github.com/Theldus/wsServer
It's works pretty well and quite RayLib-style (just a couple of .c file, written in C99), the only thing is that it doesn't support encryption yet, even tho there's an easy workaround for that.

This issue has been open for too long, many alternatives have been proposed here and actually rnet.h implementation is still there for anyone willing to review/improve it. I'm closing the issue.

Hello.
I'm the author of nbnet. I discussed with Ray and started working on a potential rnet implementation (as a nbnet wrapper). You can follow the progress here for now. Take a look at rnet.h for a good overview of the API.

@raysan5 I forked raylib: https://github.com/nathhB/raylib.
This is where I will update rnet so you can look there when you want to review it. I'm gonna leave it as-is until it has been reviewed as I think it lays a good fundation to build upon.

After reviewing rnet implementation using nbnet I got to the conclusion that nbnet is so good that a rnet wrapper does not make much sense. I checked the code and almost all rnet functions just implement a direct call to some nbnet function, maybe adding some tracelog or small check.
Other libraries used on raylib like miniaudio are more complex and having a top layer simplifying usage makes sense but nbnet already seems very easy to use, even for beginners.
Still, some nbnet usage examples will be added to the official raylib/examples/network and nbnet will be the go-to option recommended for raylib users wanting to add network features to their games. :D


Ft historical device measurements

Airqo and Kcca historical device measurements
WHAT DOES THIS PR DO?

[x] Extracts historical device measurements from KCCA and Things-speak
[x] Transforms the data to match the format required by Netmanager
[x] Loads the data into Netmanager

WHAT JIRA STORY/TASK IS RELATED TO THIS PR?

For KCCA
cd AirQo-api/src/data-mgt/python/batch-jobs/kcca
Follow instructions in the Readme.md

For AIRQO
cd AirQo-api/src/data-mgt/python/batch-jobs/airqo
Follow instructions in the Readme.md

WHICH ENDPOINTS SHOULD BE READY FOR TESTING?:
http://localhost:3000/api/v1/
IS THERE ANY JENKINS CONSOLE LINK FOR CODE COVERAGE AND BUILD INFO?
N/A
ARE THERE ANY RELATED PRs?
N/A

@NoahNsimbe @Baalmart I think it's better for this job to push directly to the events collection (mongodb). This might required more work to be done but It removes the in-between device-registry micro-service, thus reducing on possible points of failure and possible improve on the job's speed. We want our jobs to be independent too. This is just a though.

Sure @ja-odur , this is a very good thought/suggestion. We just need to have review it further by considering our system architecture that involves multitenancy and microservices.
On the side note, I do not see any significant difference if the logic behind the interface (device registry endpoint) is transferred to the actual job.
A few questions/thoughts around this thought:

When we transfer the logic from the interface (endpoint) to the job, how do we plan to handle reusability of that logic?
How do we plan to deal with multitenancy within the job.
How will the transfer of that logic from the interface to the job improve performance? Can't that performance be improved from the interface itself?
If the logic of the job is transferred to the job, will it consider the DB schema design we already have in place? Or it will have to use its own design?
What will be the scope of the job? Considerations: extract, transform and load? With the load also having more logic to enforce more DB/schema design considerations?
How will the inclusion of the schema design enforcement logic (device registry interface) in the job have on the overall performance of the job?

The kcca batch process job worked fine for me although the dates being inserted into the events collections dont match the range specified in the env.kcca.batch env file.
I tried running the airqo batch process job and got an error that looks likes a JsonDecode Error


Refactor image card tracking

What
Rework how GA4 tracking is done on the image card component (prototyped a different approach in https://github.com/alphagov/govuk_publishing_components/pull/3788)
Specifically:

don't include any tracking at all on the component 😱
refactor the component to use the component wrapper helper...
...which allows any data attributes to be passed safely to the component without overwriting existing ones
...meaning that the link tracker and required data attributes can be passed to the component where required

Pros:

another component using the component wrapper helper
no additional code for tracking on the component, all handled by passing options

Cons:

arguably a breaking change, as the current tracking for the homepage uses of the image card will stop working

This PR removes the test for passing a lang attribute to the component, as this functionality is now provided by the component wrapper helper.
Why
Part of the GA4 migration.
Visual Changes
None.
Trello card: https://trello.com/c/IymJPytQ/762-sort-out-image-card-component-tracking

@AshGDS that's correct. The problem I faced was that tracking can't be isolated on this component - it needs to be passed data for things like index_section (on the homepage at least). I had to decide between a partial implementation of tracking, with some kind of option to pass remaining options, or this. This seemed simpler and cleaner, and affords more flexibility.
I explored the alternative in https://github.com/alphagov/govuk_publishing_components/pull/3788 and it just felt messy to me, but if there's another approach I'd be happy to discuss.

@andysellick No worries - I'm happy with this approach :+1: Thanks!


how to run neural algorithm on remote image

How can we run on remote image rather than file path and integrated the code base with existing technologies like nodejs !

See https://github.com/nikos-glikis/nsfw-docker -- It has the option for the file to be a URL rather than a local file (the script downloads it to a temporary file and deletes after it's been scored).
Not quite sure what you're trying to do with nodejs but the model (to my knowledge) cannot be used in nodejs directly since it's a caffe model.


Can't no longer run contracts.tests

After latest updates, running manage.py test contracts.tests gives following error:
from django.db.models import Value, F, TextField ImportError: cannot import name 'Value'
Am I missing something?
Also in order to install psycopg2 had to install libpq-dev python-dev packages

We are requiring Django 1.8, released yesterday or so, since I upgraded django-sphinxql to allow indexing data from Foreign fields.
For psycopg2, I'm sorry, I tried on travis and Mac os and it was working fine. Is there anything we can add to readme?

It seems that this is more of problem of my system configuration and lack of general knowledge than anything else. Thanks for the help.

@nmota, you need any help (for your system conf. or general knowledge), please let me know. You  can reach me by email if you don't think it shouldn't be discussed here.

Btw, ebb1035135d11ac630cd5ccd5e96da6196526b02 updated travis' configuration to test the contracts app. There you can see what you should install to run the contracts tests. I'm closing this issue, but feel free to open if you believe something can be further improved.


Fixed duplicate holidays and some more italian nuances

As requested I fixed the italian holidays that were duplicated, also added some more italian forms

Thanks


How to fix "ValueError: Input mask to CRF must have dim 2 if not None"?

Hello,
I am trying to implement the CRF layer into a semantic segmentation a U-Net-style convolutional neural network for semantic segmentation in order to refine the predicted masks. I have looked through many projects on GitHub for a CRF implementation and this is one of the few that seem feasible for my use. However, I have come across an error that I cannot seem to overcome.
Below is how I prepared and loaded my training images and their corresponding masks that would be fed to the model:
`# Create new arrays to store testing images/masks
X_test = np.zeros((len(test_img), IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS), dtype=np.float32)
Y_test = np.zeros((len(test_masks), IMG_HEIGHT, IMG_WIDTH, 1), dtype=np.float32)
Resize augmented training images
for n, id_ in tqdm(enumerate(aug_img), total=len(aug_img)):
# Load images
img = imread('C:/Users/manos/Desktop/unetResearch/unetproto/dataset/training/images/augmented/'+id_)
img = resize(img, (IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS), mode = 'constant', preserve_range = True)
# Save images
X[n] = img/255
Resize augmented training masks
for n, id_ in tqdm(enumerate(aug_masks), total=len(aug_masks)):
# Load images
mask = imread('C:/Users/manos/Desktop/unetResearch/unetproto/dataset/training/masks/augmented/'+id_)
mask = np.expand_dims(resize(mask, (IMG_HEIGHT, IMG_WIDTH), mode = 'constant', preserve_range = True), axis=-1)
# Save masks
Y[n] = mask/255'
And this is the convolutional neural network as I try to implement the CRF layer:
`#This is the best UNET model as of now
#Encoder
sequence_input = tf.keras.layers.Input(shape=(256, 256, 3), dtype=tf.float32, name='sequence_input')
sequence_mask = tf.keras.layers.Lambda(lambda x: tf.greater(x, 0))(sequence_input)
c1 = tf.keras.layers.Conv2D(16, (3, 3), kernel_initializer='he_normal', padding='same')(sequence_input)
n1 = tf.keras.layers.BatchNormalization()(c1)
a1 = tf.keras.activations.relu(n1, alpha=0.0, max_value=None, threshold=0)
c1 = tf.keras.layers.Dropout(0.1)(a1)
c1 = tf.keras.layers.Conv2D(16, (3, 3), kernel_initializer='he_normal', padding='same')(c1)
n1 = tf.keras.layers.BatchNormalization()(c1)
a1 = tf.keras.activations.relu(n1, alpha=0.0, max_value=None, threshold=0)
p1 = tf.keras.layers.MaxPooling2D((2, 2))(a1)
c2 = tf.keras.layers.Conv2D(32, (3, 3), kernel_initializer='he_normal', padding='same')(p1)
n2 = tf.keras.layers.BatchNormalization()(c2)
a2 = tf.keras.activations.relu(n2, alpha=0.0, max_value=None, threshold=0)
c2 = tf.keras.layers.Dropout(0.1)(a2)
c2 = tf.keras.layers.Conv2D(32, (3, 3), kernel_initializer='he_normal', padding='same')(c2)
n2 = tf.keras.layers.BatchNormalization()(c2)
a2 = tf.keras.activations.relu(n2, alpha=0.0, max_value=None, threshold=0)
p2 = tf.keras.layers.MaxPooling2D((2, 2))(a2)
c3 = tf.keras.layers.Conv2D(64, (3, 3), kernel_initializer='he_normal', padding='same')(p2)
n3 = tf.keras.layers.BatchNormalization()(c3)
a3 = tf.keras.activations.relu(n3, alpha=0.0, max_value=None, threshold=0)
c3 = tf.keras.layers.Dropout(0.2)(a3)
c3 = tf.keras.layers.Conv2D(64, (3, 3), kernel_initializer='he_normal', padding='same')(c3)
n3 = tf.keras.layers.BatchNormalization()(c3)
a3 = tf.keras.activations.relu(n3, alpha=0.0, max_value=None, threshold=0)
p3 = tf.keras.layers.MaxPooling2D((2, 2))(a3)
c4 = tf.keras.layers.Conv2D(128, (3, 3), kernel_initializer='he_normal', padding='same')(p3)
n4 = tf.keras.layers.BatchNormalization()(c4)
a4 = tf.keras.activations.relu(n4, alpha=0.0, max_value=None, threshold=0)
c4 = tf.keras.layers.Dropout(0.2)(a4)
c4 = tf.keras.layers.Conv2D(128, (3, 3), kernel_initializer='he_normal', padding='same')(c4)
n4 = tf.keras.layers.BatchNormalization()(c4)
a4 = tf.keras.activations.relu(n4, alpha=0.0, max_value=None, threshold=0)
p4 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(a4)
c5 = tf.keras.layers.Conv2D(256, (3, 3), kernel_initializer='he_normal', padding='same')(p4)
n5 = tf.keras.layers.BatchNormalization()(c5)
a5 = tf.keras.activations.relu(n5, alpha=0.0, max_value=None, threshold=0)
c5 = tf.keras.layers.Dropout(0.3)(a5)
c5 = tf.keras.layers.Conv2D(256, (3, 3), kernel_initializer='he_normal', padding='same')(c5)
n5 = tf.keras.layers.BatchNormalization()(c5)
a5 = tf.keras.activations.relu(n5, alpha=0.0, max_value=None, threshold=0)
Decoder
u6 = tf.keras.layers.Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(a5)
u6 = tf.keras.layers.concatenate([u6, a4])
c6 = tf.keras.layers.Conv2D(128, (3, 3), kernel_initializer='he_normal', padding='same')(u6)
n6 = tf.keras.layers.BatchNormalization()(c6)
a6 = tf.keras.activations.relu(n6, alpha=0.0, max_value=None, threshold=0)
c6 = tf.keras.layers.Dropout(0.2)(a6)
c6 = tf.keras.layers.Conv2D(128, (3, 3), kernel_initializer='he_normal', padding='same')(c6)
n6 = tf.keras.layers.BatchNormalization()(c6)
a6 = tf.keras.activations.relu(n6, alpha=0.0, max_value=None, threshold=0)
u7 = tf.keras.layers.Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(a6)
u7 = tf.keras.layers.concatenate([u7, a3])
c7 = tf.keras.layers.Conv2D(64, (3, 3), kernel_initializer='he_normal', padding='same')(u7)
n7 = tf.keras.layers.BatchNormalization()(c7)
a7 = tf.keras.activations.relu(n7, alpha=0.0, max_value=None, threshold=0)
c7 = tf.keras.layers.Dropout(0.2)(a7)
c7 = tf.keras.layers.Conv2D(64, (3, 3), kernel_initializer='he_normal', padding='same')(c7)
n7 = tf.keras.layers.BatchNormalization()(c7)
a7 = tf.keras.activations.relu(n7, alpha=0.0, max_value=None, threshold=0)
u8 = tf.keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(a7)
u8 = tf.keras.layers.concatenate([u8, a2])
c8 = tf.keras.layers.Conv2D(32, (3, 3), kernel_initializer='he_normal', padding='same')(u8)
n8 = tf.keras.layers.BatchNormalization()(c8)
a8 = tf.keras.activations.relu(n8, alpha=0.0, max_value=None, threshold=0)
c8 = tf.keras.layers.Dropout(0.1)(a8)
c8 = tf.keras.layers.Conv2D(32, (3, 3), kernel_initializer='he_normal', padding='same')(c8)
n8 = tf.keras.layers.BatchNormalization()(c8)
a8 = tf.keras.activations.relu(n8, alpha=0.0, max_value=None, threshold=0)
u9 = tf.keras.layers.Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same')(a8)
u9 = tf.keras.layers.concatenate([u9, a1], axis=3)
c9 = tf.keras.layers.Conv2D(16, (3, 3), kernel_initializer='he_normal', padding='same')(u9)
n9 = tf.keras.layers.BatchNormalization()(c9)
a9 = tf.keras.activations.relu(n9, alpha=0.0, max_value=None, threshold=0)
c9 = tf.keras.layers.Dropout(0.1)(a9)
c9 = tf.keras.layers.Conv2D(16, (3, 3), kernel_initializer='he_normal', padding='same')(c9)
n9 = tf.keras.layers.BatchNormalization()(c9)
a9 = tf.keras.activations.relu(n9, alpha=0.0, max_value=None, threshold=0)
crf = CRF(2)
outputs = crf(a9, mask=sequence_mask)`
But I receive an error: "ValueError: Input mask to CRF must have dim 2 if not None"
Can someone please help me figure out a solution?

The sequence_mask has dim 3 (256, 256, 3). However, input mask to CRF must have dim 2.
And the inputs to CRF must have rank 3, but your inputs have rank 4.

@luozhouyang can you deploy a functional example ?

@luozhouyang I am working with sequence classification (same structure as NER), my sequence mask has 3 dim (#batches, #words, #dims) how can I reduce it to 2d to fit your crf ?
the input to crf has 3 dims (#batches, #words, #classes).

maybe the summary can help
Model: "model_5"

Layer (type)                 Output Shape              Param #
input_10 (InputLayer)        [(None, 10, 100)]         0

time_distributed_8 (TimeDist (None, 10,<PHONE_NUMBER>

bidirectional_9 (Bidirection (None, 10,<PHONE_NUMBER>

time_distributed_9 (TimeDist (None, 10, 6)             3102

masking_3 (Masking)          (None, 10, 6)             0
Total params: 3,903,382
Trainable params: 3,903,382
Non-trainable params: 0

hey guys, sorry for taking so long to reply to you. I  refactored the code, the error you mentioned above is solved. You can found example in README.

Hi, I just tested the updated version, I am getting now
 ValueError: Shape must be rank 2 but is rank 3 for '{{node cond/Slice}} = Slice[Index=DT_INT32, T=DT_INT32](cond/add_1/Cast, cond/Slice/begin, cond/Slice/size)' with input shapes: [?,10,6], [2], [2].

Can you paste your code that build the Model?

embedding_layer = Embedding(len(word_index) + 1, embedding_dim, weights=[embedding_matrix],
                            input_length=maxlen, trainable=False)

sentence_input = Input(shape=(maxlen,), dtype='int32')
embedded_sequences = embedding_layer(sentence_input)
lstm_word = Bidirectional(GRU(258, return_sequences=True))(embedded_sequences)
attn_word = HierarchicalAttentionNetwork(100)(lstm_word)
sentenceEncoder = Model(sentence_input, attn_word)

review_input = Input(shape=(max_sentences, maxlen), dtype='int32')

review_encoder = TimeDistributed(sentenceEncoder)(review_input)
lstm_sentence = Bidirectional(LSTM(258, return_sequences=True))(review_encoder)

final_dense = TimeDistributed(Dense(n_tags, activation="relu"))(lstm_sentence)

base = Model(review_input, final_dense)
model = CRFModel(base, n_tags)

# no need to specify a loss for CRFModel, model will compute crf loss by itself
model.compile(
    optimizer=tf.keras.optimizers.Adam(3e-4),
    metrics=['acc'],
    )

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_8 (InputLayer)         [(None, 10, 100)]         0
_________________________________________________________________
time_distributed_5 (TimeDist (None, 10,<PHONE_NUMBER>
_________________________________________________________________
bidirectional_7 (Bidirection (None, 10,<PHONE_NUMBER>
_________________________________________________________________
time_distributed_6 (TimeDist (None, 10, 6)             3102
=================================================================
Total params: 3,903,382
Trainable params: 2,211,782
Non-trainable params: 1,691,600
_____________________________________________________________

Can you put the reproducible code in Colab, and paste the share link here?

https://colab.research.google.com/drive/1xKH0sRljeoRnyf16_lODQGHsIALHfER0
Please download the data from GitHub link mentioned in notebook

labels are not need to one-hot encoding

ok, I will try without one hot encoding
pt., 7 maj 2021 o 10:18 luozhouyang @.***> napisał(a):

labels are not need to one-hot encoding
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/luozhouyang/keras-crf/issues/1#issuecomment-834163171,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA35VLXQWHD3AQ5TZ7LGBL3TMOO4BANCNFSM4XXTNZIQ
.

--
URSZULA CZERWINSKA
Data Scientist, Ph. D.
http://urszulaczerwinska.github.io/about/

Just change these lines:
# labels = [to_categorical(list(lab), num_classes=6) for lab in new_labs]
labels = new_labs
print(labels[0])
print(labels[0:3])
labels[0]
labels = np.array(labels)

And you can get training results like this:

This accuracy is really low. The results without crf are 0.86
https://colab.research.google.com/drive/10moAebf-fbxyCCTg69-PVjjoQw4T3D5h

And with a different (less elegant) crf implantation  0.88
https://colab.research.google.com/drive/1usnVjTRxEYrudo89yx2gtjZ_rBhNiM7i

I adapted the code as you say.
I added zero_mask=True to mask the padding.
I have one more question. Why there are predictions for the "padded" values in Y?
For instance:
y_test[0] = array([1, 1, 3, 3, 3, 4, 0, 0, 0, 0], dtype=int32)
y_pred[0] = array([1, 1, 1, 2, 3, 4, 3, 3, 3, 3])

Is that expected behavior?
https://colab.research.google.com/drive/1xKH0sRljeoRnyf16_lODQGHsIALHfER0?usp=sharing

Just change these lines:
# labels = [to_categorical(list(lab), num_classes=6) for lab in new_labs]
labels = new_labs
print(labels[0])
print(labels[0:3])
labels[0]
labels = np.array(labels)

And you can get training results like this:


always force the cfg to show

When I run this on a random ruby file without a typed sigil, it shows a white page. Can we always just force the pass to run by typing the file?
Motivation

Test plan

See included automated tests.

Yeah, I don't think that matters here


reintroduce user management into defaults and add testing user creation

Seems like there are use cases where node_exporter might be needed to be run as a different user than one which this role creates.
Only change in a logic is to skip user management (user and group creation) when node_exporter is expected to be running as root. Otherwise this just moves variables to make setting user visible and documented. Additionally I put some tests around user and group creation.
Spawned by https://github.com/prometheus/node_exporter/issues/1246

I really don't like that there's a feature in node_exporter that requires root. :frowning:


Why custom ThreadFactory is slow ?

Netty version: 4.1.18.Final
JVM version: Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
OS version: Windows 10
new NioEventLoopGroup(0, new io.netty.util.concurrent.DefaultThreadFactory(NioEventLoopGroup.class))
is slow
jmh:
Warmup Iteration   2: 71.169 ops/ms
Warmup Iteration   3: 80.204 ops/ms
Warmup Iteration   4: 85.780 ops/ms
Warmup Iteration   5: 81.681 ops/ms
Iteration   1: 84.452 ops/ms
Iteration   2: 84.801 ops/ms
Iteration   3: 85.283 ops/ms
Iteration   4: 85.253 ops/ms
Iteration   5: 84.905 ops/ms
eventLoopGroup = new NioEventLoopGroup(0);
is fast
jmh:
Warmup Iteration   2: 74.367 ops/ms
Warmup Iteration   3: 98.894 ops/ms
Warmup Iteration   4: 102.583 ops/ms
Warmup Iteration   5: 98.344 ops/ms
Iteration   1: 105.881 ops/ms
Iteration   2: 106.302 ops/ms
Iteration   3: 107.134 ops/ms
Iteration   4: 107.732 ops/ms
Iteration   5: 107.150 ops/ms
everything else is the same
what's the reason ?

Please share the benchmark code

Am 22.12.2017 um 10:30 schrieb Hank<EMAIL_ADDRESS>Netty version: 4.1.18.Final
JVM version: Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
OS version: Windows 10
new NioEventLoopGroup(0, new io.netty.util.concurrent.DefaultThreadFactory(NioEventLoopGroup.class))
is slow
jmh:
Warmup Iteration 2: 71.169 ops/ms
Warmup Iteration 3: 80.204 ops/ms
Warmup Iteration 4: 85.780 ops/ms
Warmup Iteration 5: 81.681 ops/ms
Iteration 1: 84.452 ops/ms
Iteration 2: 84.801 ops/ms
Iteration 3: 85.283 ops/ms
Iteration 4: 85.253 ops/ms
Iteration 5: 84.905 ops/ms
eventLoopGroup = new NioEventLoopGroup(0);
is fast
jmh:
Warmup Iteration 2: 74.367 ops/ms
Warmup Iteration 3: 98.894 ops/ms
Warmup Iteration 4: 102.583 ops/ms
Warmup Iteration 5: 98.344 ops/ms
Iteration 1: 105.881 ops/ms
Iteration 2: 106.302 ops/ms
Iteration 3: 107.134 ops/ms
Iteration 4: 107.732 ops/ms
Iteration 5: 107.150 ops/ms
everything else is the same
what's the reason ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

netty-benchmark.zip

key file:
netty-server:benchmark.rpc.Server
netty-client:benchmark.rpc.netty.client.NettyClientConnector

@hank-whu can you just push it to GitHub as a repository so its easier to browse and comment ?

registering my interest as we use a custom ThreadFactory.

https://github.com/hank-whu/rpc-benchmark

@hank-whu honestly it should not make any difference as you see in our source we also just use DefaultThreadFactory if you not specify one.

yes, it should be same,
but can you run the test in your computer ?

@hank-whu can you add details how to exactly run it etc ? I am on a Mac

@normanmaurer it's very easy:
1.cd benchmark-base; mvn install
2.cd netty-server; mvn clean; mvn test
3.cd netty-client; mvn clean; mvn test
then change the source

@hank-whu what if you create the thread factory outside of the benchmark? eg. in the jmh state.

@johnou what's your meaning, can you show me the code?
this is the source code:
https://github.com/hank-whu/rpc-benchmark

sorry I can not reproduce... Please update to latest version and verify again. If you still thing there is an issue please re-open


2.0.2 SNAPSHOT r5c38483 possibly reports incorrect file and directory scan count

using this version to import  a total of 195614 folders, 565921 pdf files and 772691 properties.xml files
The statistics reported on the job status screen appears to over-state the number of files and directories scanned.
Status:     Succeeded
Initiating User:    admin
Source Name:    Default
Source directory    /import/Company Home/User Homes
Target Space:   /Company Home/User Homes
Import Type:    Streaming
Dry run:    No
Batch Weight:   100
Threads:    0 active of 0 total
Start Date:     2016-07-12T12:25:51Z
Scan End Date:  2016-07-12T15:14:21Z
End Date:   2016-07-12T16:47:39Z
Scan Duration:  02h 48m 29s 530.760ms
Duration:   04h 21m 47s 459.256ms
Currently Importing:

Source (read) Statistics
Directories scanned:    391220  38.698 / sec
Files scanned:  2699544 267.03 / sec

Target (write) Statistics
Aspects associated: 6778444 431.543 / sec
Batches completed:  7728    0.492 / sec
Batches submitted:  7728    0.492 / sec
Bytes imported:<PHONE_NUMBER>7 2309012.5 / sec
Content streamed:   577081  36.739 / sec
In place content linked:    0   0 / sec
Metadata properties imported:   14374735    915.153 / sec
Nodes imported: 772691  49.193 / sec
Nodes skipped:  0   0 / sec
Out-of-order retries:   0   0 / sec
Versions imported:  0   0 / sec

the final node  count reported (772691) is correct
2016-07-12 12:47:39,350  INFO  [bulkimport.impl.LoggingBulkImportCompletionHandler] [BulkImport-Scanner] BULKIMPORT: Streaming bulk import completed (Succeeded) in 04h 21m 47s 459.256ms.
Batches:                7728 imported of 7728 submitted (0.492 / sec)
Nodes:                  772691 (49.193 / sec)
Bytes:<PHONE_NUMBER>7 (2309012.500 / sec)
Versions:               0
Metadata properties:    14374735
Files:                  0 in-place, 577081 streamed, 0 skipped
Out-of-order batches:   0

(previously reported in #55)

The source content set is scanned twice - once for folders and once for files, which explains most of what you're seeing (195614 X 2 = 391228, which is close to 391220).  The delta of 4 (8 ÷ 2) may be due to permissions (files & folders the tool doesn't have permission to read are skipped, with a warning in the log, although that may not be showing up given the odd logging behaviour we saw in issue #55), or may be due to the way you're counting the folders on-disk (the root folder that you specify in the UI, for example, is not counted by the tool).
I'll add this as an item to the FAQ..

Addressed in this FAQ item.


ROBUST HYPORADISE DATASET training splits

Hello,
Great work! I wanted to ask for the  ROBUST HYPORADISE DATASET training splits. I checked the HF page linked in README, and it looks like only the test sets were released. Any help would be appreciated!
Thank You again!

Hi, Thanks for interest in our work. Sorry for inconvenience.
The training split is large and thus now still in package, i'll release the link to drive in hf once available, hope can complete next month.

@YUCHEN005 let's release the RobustGER training script in Huggingface with large space limit.

@YUCHEN005 Any ETA on the train split?

Hi all, sorry for delay, please see our new update in huggingface: https://huggingface.co/datasets/PeacefulData/Robust-HyPoradise

this issue can be closed. @YUCHEN005


Issue#2550 - Fixed getGeneratedKeys functionality for execute API

Description:
The execute API for INSERT was not reading off the explicit TDS_DONE token that actually contains the update count returned by the server.  As a result, it was not able to correctly report back the update count for getUpdateCount and also the subsequent resultsets on getMoreResults.
Note that this works correctly for excecuteUpdate API.
Testing:

All existing tests pass
Introduced a number of tests testing various combinations of execute/executeUpdate API, INSERT/UPDATE/DELETE/MERGE DML operations, followed by explicit SELECT statement or getGeneratedKeys call.

Internal tests pass


[WIP] Deploy jupyterhub-ssh

TODOs from https://github.com/2i2c-org/pangeo-hubs/issues/4:
Phase 1: Get jupyterhub-ssh working

[x] Get latest version of jupyterhub-ssh (published in this YAML file)
[x] Add it to hub/requirements.yaml so it gets pulled in as a dependency
[x] Set jupyterhub-ssh.hubUrl to full URL for each hub (deployments/ohw/config/staging.yaml, deployments/ohw/config/prod.yaml, deployments/farallon/config/staging.yaml, deployments/farallon/config/prod.yaml)
[x] Generate a private key for each hub (with ssh-keygen -f <some-name>), and copy it to jupyterhub-ssh.hostKey in the secret files for each hub (deployments/ohw/secrets/staging.yaml, etc). Use sops <filename> to edit the secret file - it will open in an editor.
[x] (temporarily) set ssh.service.type to LoadBalancer to get an external IP
[x] Disable sftp (temporarily) by setting sftp.enabled to false
[x] Deploy this to staging, and test out sshing in with the public IP the load balancer gets!

Phase 2: Get jupyterhub-ssh working with the same URL as hub
Instead of sshing in with a separate domain / URL, we want users to ssh in with the same URL as the hub. This is doable since ssh listens on port 22, and traefik can proxy that too.

[x] Tell the proxy-public service to accept traffic on port 22 as well, by specifying jupyterhub.proxy.extraPorts - https://github.com/berkeley-dsep-infra/datahub/blob/staging/hub/values.yaml#L49. Should target port 8022, since we'll make traefik listen on that port for proxying to our jupyterhub-ssh service
[x] Tell traefik to proxy incoming traffic on port 8022 to the jupyterhub-ssh service on port 22. See sections under jupyterhub.proxy.traefik.extraDynamicConfig.tcp in https://github.com/berkeley-dsep-infra/datahub/blob/staging/hub/values.yaml#L78.
[x] Allow ingress traffic from traefik into our ssh servers, by adding these labels to our traefik pod
[x] Allow ingress traffic into traefik from the outside, with these network policy changes

This should let you ssh in with the same domain as the hub!
Once done, you wanna clean up the external IP for the ssh service:

[x] Set jupyterhub-ssh.ssh.service.type to 'ClusterIP. You might have to manually delete the jupyterhub-ssh service before deploying this, since helm might not like downgrading from LoadBalancer to ClusterIP type service
Closes https://github.com/2i2c-org/pangeo-hubs/issues/4

I believe I ticked all the boxes on the todo list @yuvipanda :tada:
I have also deployed this on ohw staging and it looks like it's working properly.
One thing I noticed is that if I don't have the notebook server started before I ssh into the hub, I see this timeout error:
Starting your server.............................................................failed to start server on time!

I was thinking this might be because of a timeout set up somewhere in jupyterhub-ssh or because when starting the server, you have the 2 resources choices. Either way, if the server is started beforehand and then ssh, everything works.
Also, do we want to also enable the sftp service for the hubs?

@GeorgianaElena \o/ THAT IS AWESOME! 🎉 🥳

because when starting the server, you have the 2 resources choices.

I think this is the reason, most likely. If you check the hub logs you should be able to tell.

Also, do we want to also enable the sftp service for the hubs?

Yeah, but let's finish this one up. I think we should:

Deploy this to all 4 hubs
Merge this PR
Add a little user documentation on how people can use it (not sure where?)
Tell folks in gitter!

How does that sound?

How does that sound?

Perfect :1st_place_medal:

Add a little user documentation on how people can use it (not sure where?)

I could set up the RTD docs for this project and add it there, alongside the info we already have in the readme.
Or add to the readme... But I believe it will make it a bit cluttered and the info, find to hard.
What do you think?

I could set up the RTD docs for this project and add it there, alongside the info we already have in the readme.
Or add to the readme... But I believe it will make it a bit cluttered and the info, find to hard.
What do you think?

We can start adding it to the README, and add an issue on RTD?

jupyterhub-ssh is now working for all four hubs :partying_face:, so I'll go ahead and merge this PR.
I will add some instructions to the readme on how to use it so that folks can test it, but I opened up an issue about setting up RTD as you suggested.
TY @yuvipanda :heart:


feat: add new footer links

closes: #693

:tada: This PR is included in version 1.21.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Improve code generation

There's currently an explosion in the number of parameters passed to the generated code from Python. This obfuscates the readability of the generated code itself. Instead of passing each parameter, one by one, we could rather use C structs and then generated stuff like "x.start, x.end, ..."

 Closing.


fix(zopafooter): copy change to press office button

Change button copy from Press Office -> Press office

:tada: This PR is included in version 4.45.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add reference to 'Environment planning' page

The section about how many sandboxes are needed is very short. Maybe add a reference to the 'Environment planning' page (https://docs.microsoft.com/en-us/dynamics365/unified-operations/fin-and-ops/imp-lifecycle/environment-planning).
Using the Tier-2 for both configuration and UAT doesn't seem realistic to me.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cd0c5515-aa2a-64c2-e5b6-093eecdfff53
Version Independent ID: 53446ef1-fc30-9ad1-d097-409fcd5c7949
Content: Development and continuous delivery FAQ - Finance & Operations | Dynamics 365 | #MSDyn365FO
Content Source: articles/dev-itpro/dev-tools/continuous-delivery-faq.md
Service: dynamics-ax-platform
Product: ****
Technology: ****
GitHub Login: @RobinARH
Microsoft Alias: robadawy

@JeroenLeeflang, thanks for posting here!  I've added the link, as you suggested -- that update will be published later today.
@robadawy, can you please review the comment related to Tier-2.  Please update the topic if needed.

Bug 377134 has been logged to track this issue.

Closing, as the link has been added as suggested.
I discussed the Tier-2 comment with a PM, and he feels that the article sufficiently identifies you can get more Tier-2 environments to support your needs.


Layout and Text Detection Discrepancy Between Docker Image (PyTorch) and Hugging Face Live Version

Bug 💥
I have run the docker image, pytorch version to detect layout and text in the image and pdf, but it detects only two tables and titles, while in the live version deployed on huggingface, it detects well.
Expected behavior  🧮
I want to have same layouts as detected in live version.
Screenshots 🖼
Images are private
**Desktop (Linux)

https://huggingface.co/spaces/deepdoctection/deepdoctection/discussions/3


Some datatypes (like NULL) crash the process

I have not had time to really dig into this, but this is what I experience at the moment.
When I do SQL queries with bound parameters that are containing a null value I get a crash in request.js line 153. The error seems to be that some data types (like NULL) doesn't have a validate function available.
I'm using version 1.12.3
This is the function:
Request.prototype.validateParameters = function () {
    var i, len, parameter, ref, value;
    ref = this.parameters;
    for (i = 0, len = ref.length; i < len; i++) {
      parameter = ref[i];
      value = parameter.type.validate(parameter.value); // <---- ERROR HERE
      if (value instanceof TypeError) {
        return this.error = new RequestError("Validation failed for parameter '" + parameter.name + "'. " + value.message, "EPARAM");
      }
      parameter.value = value;
    }
    return null;
  };

Can you post the complete error + backtrace?

I hope this is what you want:
c:\Klocktornet\DISEServerWebUI\node_modules\any-db-mssql\node_modules\tedious\lib\request.js:154
      value = parameter.type.validate(parameter.value);
                             ^
TypeError: undefined is not a function
    at Request.validateParameters (c:\Klocktornet\DISEServerWebUI\node_modules\any-db-mssql\node_modules\tedious\lib\request.js:154:30)
    at Request.transformIntoExecuteSqlRpc (c:\Klocktornet\DISEServerWebUI\node_modules\any-db-mssql\node_modules\tedious\lib\request.js:90:14)
    at Connection.execSql (c:\Klocktornet\DISEServerWebUI\node_modules\any-db-mssql\node_modules\tedious\lib\connection.js:962:13)
    at _execNextInQueue (c:\Klocktornet\DISEServerWebUI\node_modules\any-db-mssql\index.js:578:11)
    at process._tickCallback (node.js:355:11)

The parameter variable was containing this:
{ type: { type: 'NULL', name: 'Null', id: 31 },
  name: 'p2',
  value: null,
  output: false,
  length: undefined,
  precision: undefined,
  scale: undefined }

How did you assign this parameter to the request? There's something wrong with tedious here, but a reproducible test case would help me a long way.

I'm using a library called any-db-mssql (that I had to do some local modifications to).
The library is adding the parameter as this:
request.addParameter(keys[i], type, value, options);

Where the function parameters are:
console.log('addParameter', keys[i], type, value, options);

// Output:
addParameter p2 { type: 'NULL', name: 'Null', id: 31 } null null

A working type seems to look more like this:
addParameter p1 { type: 'NVARCHAR',
  name: 'NVarChar',
  hasCollation: true,
  dataLengthLength: 2,
  maximumLength: 4000,
  declaration: [Function],
  resolveLength: [Function],
  writeTypeInfo: [Function],
  writeParameterData: [Function],
  validate: [Function],
  id: 231 } value null

Maybe this is  an error in the calling library that is triggering wrong behavior?

Weird. I don't think tedious currently accounts for people using tedious.TYPES.Null as a parameter type. I'll dig into this once I find some free time!

Ok, thanks :+1:
If you know of an alternative way of setting a NULL value please feel free to inform me :)

If you know of an alternative way of setting a NULL value please feel free to inform me :)

You can just use the actual type that the parameter would have if it's not null, and then pass null as the actual value.

@arthurschreiber @Hadis-Fard  I think we should deprecate TYPES.Null too. Since NULL not a SQL Server data type, the sp_executesql query formed with null as type will run into syntax error (in Server side).
plus, from TDS spec,

NULLTYPE can be sent to SQL Server (for example, in RPCRequest), but SQL Server never emits NULLTYPE data.

What do you think?


Delete pr branch test

testing pr workflow.

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-nodejs-mongo
Remote: azure-samples
Branch: pr/21
You can initialize this project with:
azd init -t kaizentm/todo-nodejs-mongo -b pr/21

View Changes | Compare Changes

Project: todo-python-mongo
Remote: azure-samples
Branch: pr/21
You can initialize this project with:
azd init -t kaizentm/todo-python-mongo -b pr/21

View Changes | Compare Changes


Needs help on TcpClient and TcpListener

As mentioned in ToDo list "Something similar to TcpClient and TcpListener", are
you working on this ? If yes when can we expect it ?

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 27 Nov 2012 at 4:25

I haven't done any real work on the project for a few years, so it is not likely this will ever be implemented. Closing the issue.


Part 6: enable windows unit test

First commit is marking a few tests as flaky.
Second commit enables unit testing for windows.
Depends on:

#3550
#3535
#3553

Changes to make the tests pass (besides above PRs):

skipping a few tests that heavily imply linux
skip test for apparently broken condition (#3554)
change test to match code (TestParseVolumeOptions)

See inline comments.
Leaving as draft until the dependencies are merged.

Re-run failures are:
TestLogsWithInheritedFlags for Docker - already spotted as flaky with Docker before - is the test broken or is docker broken?
TestAttach: despite a lot of efforts to tweak delays, this keeps failing again
Note that these are failing in-spite of the retries ^

Failure is fixed with #3563

Geez.
Closing was not what I meant.
Will reopen a new one.


FormDataParam default to null?

I have a method that accepts type Multipart Form Data. I want to be able to hit this endpoint sometimes without the specified @FormDataParam("file"), but right now when I leave it out the method returns a 400 bad request immediately. Is there a way I can set it up so that I can leave it out? Or is there some way I can set a default value for this (i.e. to null?). I wanted to try @DefaultValue(null) but the compiler didn't like that. Any help would be appreciated. My method declaration is below:
@POST
@Path("/publish")
@Consumes(MediaType.MULTIPART_FORM_DATA)
public Response publish(@Auth Key key,
                          @QueryParam("email") String email,
                          @HeaderParam("password") String password,
                          @QueryParam("type") PublishType type,
                          @QueryParam("message") String message,
                          @FormDataParam("file") InputStream inputStream,
                          @FormDataParam("file") FormDataContentDisposition
                              contentDispositionHeader,
                          @FormDataParam("title") @DefaultValue("") String videoTitle) {
// code here
}

What would be the @DefaultValue of a stream? What do you expect to happen?

@lutzhorn I would expect the @DefaultValue of a stream to be null. In the case that I don't want to provide a value for it, I want it to be empty and to tell me that it is empty. I would like the chance to use the null value myself before it automatically returns a 400.

@DefaultParam requires a String as value:
@Target({ElementType.PARAMETER, ElementType.METHOD, ElementType.FIELD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface DefaultValue {

    /**
     * The specified default value.
     */
    String value();
}

What would it mean to write something like this?
@DefaultValue("foo") @FormDataParam("param1") InputStream param1

The string foo as a value for an InputStream makes no sense. I don't think what you want to do is possible using these annotations.
BTW, this question is not a Dropwizard question. It is about JAX-RS and com.sun.jersey.contribs:jersey-multipart. I suggest you close this issue here and check at Jersey.

I've opened an issue at Jersey but I'm not hopeful for a response, the response rate for issue questions there seem low. I suppose I can try on StackOverflow too. Thanks for the help, @lutzhorn.

I agree that this would be a jersey issue, though I wonder if there is a workaround involving optionals:
@FormDataParam("file") Optional<InputStream> inputStream,

I'm guessing it'll require a custom jersey body provider but it may be worth investigating 🤔

@nickbabcock Good idea! I may look into that, I think that would be useful to have in other situations as well.


[Feature] Twitch OAuth provider

Hi! I've added this provider to my personal project and thought it'd be a good addition here. The implementation is the same shape as the other two.
Notable change: I had to slightly modify sendRequest as Twitch requires the Client-ID to be set in the header to fetch the user info.
I've set this PR as a draft as I assume documentation would be wanted before a merge.

Hi, thanks for opening a PR!
The code is looking good, can you open write the docs for it?

good to know, thanks!

I've added the documentation for the most part. It's mainly just a copy of the github version with the details of twitch filled in.

looks good to me,
final thing: can you update the version in package.json for @lucia-auth/oauth to 0.1.4?

Thanks for the PR, merged!!


Add Localization feature

Hi Ivan, first of all, great job.
Based on this architectural proposal, how would you suggest adding localization capability to this project? I'm asking it because APS.NET core provides great support for localization, but when I thinking about Domain Layer, Application Layer it is not so clear to me. My questions are:
1 - Where the resource files should be?
2 - How we can resolve the domain model and value objects?
3 - How would you solve it thinking in tests, business rules?
Thank you.

Hi @luizgustavogp!
Interesting question! I think I will implement it soon.
Do you mean localizing the messages in Value Objects and Error Messages? I won't go for it.
Instead, I suggest to Localize the Presentation layer. In this case, you could Localize the Presenters objects by injecting a service that have access to Resource files.

Resources should live in the Presentation layer (WebAPI).
To localize messages I would replace all strings with my own LocalizedString value object, which to be instantiated it would require ID and Text values. Then when the I need to display the presenter would convert the LocalizedString to the localized version. What do you think?
Messages should not affect tests, could you give me an example?

Regards,

@all-contributors please add @luizgustavogp for design and tests

Hi @luizgustavogp,
I have been thinking about your question for the last days. I personally don't like FluentValidation for Domain Objects like Value Objects, Entities or Aggregates because I have total control on how to design them.
I think FluentValidation is a great tool when you need to Validate an object that you don't have control like a Request Object to a WCF service, you can add this validation extension with small effort.
Going back to the original question, if you want to Localize code like the following:
IAccount account = await _accountRepository.Get(input.AccountId);
if (account == null)
{
    _outputHandler.Error($"The account {input.AccountId} does not exist or is already closed.");
    return;
}

IDebit debit = account.Withdraw(input.Amount);

if (debit == null)
{
    _outputHandler.Error($"The account {input.AccountId} does not have enough funds to withdraw {input.Amount}.");
    return;
}

Then create Resource file on each assembly, like one for Application, one for Domain, WebApi and Infrastructure. I think it is a not intrusive solution.
Best,
Ivan

Hi @ivanpaulovich , thanks for thinking about that. I see. It makes sense.
I thought of creating one resource file per layer so as not to create too many dependencies.
Thank you Ivan for collaborating with the community.


Where are the Box2D files expected to be? What files are expected?

As someone with no C++ experience this has been rather confusing... what exactly is expected of the Box2D dependency regards its location and its contents? I've tried pointing to it locally w/ BOX2D_INCLUDE_PATH as well as putting what I think are the correct Box2D contents in /usr/local/include/ to no avail. Help appreciated :-)
macOS, Rust 1.21.0

@SamHH Hi, sorry about the confusion. Both the dynamic Box2D library and the Box2D header files are expected to be accessible. The dynamic library is needed at runtime to call Box2D functions while the header files are describing the Box2D API and are needed for compilation. I don't know how it works on macOS exactly but on Linux the easiest way is to simply install Box2D on your system with your package manager or directly from source.
Hope this helps, don't hesitate if you need more information :)

Unlike all the others I'd found, those build steps actually worked perfectly... thank you!


HTML escaped dictionary cannot open in the data viewer

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

Debug a file with this code

import numpy as np
import pandas as pd
from pandas import Series
dict2 = {'a> <a href="bobs"/>':[1, 2, 3], '<b':[4, 5, 6]}
df6 = pd.DataFrame.from_dict(dict2)
print(df6) # Set breakpoint here

Attempt to view the df6 or dict2 in the data viewer

Actual behavior
Error parsing json
Expected behavior
Viewer opens

Your Jupyter and/or Python environment
Please provide as much info as you readily know

Jupyter server running: Local | Remote | N/A
Extension version: 20YY.MM.#####-xxx
VS Code version: #.##
Setting python.jediEnabled: true | false
Setting python.languageServer: Jedi | Microsoft | None
Python and/or Anaconda version: #.#.#
OS: Windows | Mac | Linux (distro):
Virtual environment: conda | venv | virtualenv | N/A | ...

Python Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

We should be able to close this. IIRC we used the rawString property on custom evaluate requests to ensure the debugger doesn't send us escaped strings.


InvalidArgumentError (see above for traceback): Invalid content in /dev/shm/wqy/model_zhzh/vocab.en: empty line found at position 4279.

I have this problem when trying to use a shared vocab. The vocab contains Chinese and English characters. And both my input and target contain Chinese characters. Does anyone  know how to solve this problem?

ok ,in question answer pairs there should not be an empty line as the numbers of questions should be the same as the number of answers.nmt system is just a query answer system presented as a language translation system i.e what would a banana mean in french---given a banana and team what do we expect in vietnamese given these conditions.Kindly match those pairs.


feat: add write_bytes for GenericBinaryBuilder

Similar to impl Write for GenericStringBuilder.
We found this helps in reducing one allocation when append bytes in many calls (compared with gather those bytes into a Vec first).
Not sure if there is some trait suitable for this method but just add it first.
cc @alamb @waynexia
The doc tests cover the basic usage.

Try impl io::Write interface. No sure if a good fit.

@andylokandy Thanks for your review! Addressed.

From the API level: this looks kind of misleading to me. The proposed Write::write or write_bytes makes BinaryBuilder closer to an opaque byte container, shadowing the concept of item and offset. I.e., array built from this API would contain only one element. Plus we don't have an API to "cut" the bytes buffer, but need to call builder.append_value(""); which looks a bit hack.
How about considering something like Vec::from_raw_parts, which accepts different buffers obviously and hence avoids the potential misunderstand?

Thank you @tisonkun and @tustvold


Development: Implement multiple exercise selection in LTI deep linking service

Checklist
General

[x] I tested all changes and their related features with all corresponding user types on a test server.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Server

[x] Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
[x] I followed the coding and design guidelines.
[x] I added multiple integration tests (Spring) related to the features (with a high test coverage).
[x] I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
[x] I documented the Java code using JavaDoc style.

Client

[x] Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data.
[x] I followed the coding and design guidelines.
[x] Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
[x] I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
[x] I added authorities to all new routes and checked the course groups for displaying navigation elements (links, buttons).
[x] I documented the TypeScript code using JSDoc style.
[x] I added multiple screenshots/screencasts of my UI changes.
[x] I translated all newly inserted strings into English and German.

Motivation and Context

This update introduces the capability to add multiple exercises in a single LTI deep-linking service response. Previously, instructors had to add exercises individually. By enabling batch addition of exercises, this change enhances usability, and reduces the administrative burden on instructors. Additionally, it includes an improved arrangement of the exercises list view, further optimizing the overall workflow and user interface.
Description

Multiple exercises can be selected to Link to external LMS (Moodle).
If the Artemis exercise is a graded or bonus exercise, it will be added to the grade book of the external LMS.
If the Artemis exercise is not a graded exercise, it won't appear on the grade book of the external LMS.
Exercise selection page rearranged:

Course title and semester information removed
Exercise start date added
Exercise graded information added

Steps for Testing

Prerequisites:

Test on Test Server 11 where lti profile is enabled.
1 Instructor

Login to Moodle (https://moodle.ase.in.tum.de/) using instructor credentials (e.g. artemis_test_user_11)
Select "LTI Testing Course"
Enable edit mode (on top right corner of the page)
Select add an activity or resource
Select "External Tool"
You will be directed to editing external tool page
Under preconfigured tools select "Artemis - https://artemis-test11.artemis.in.tum.de/" and click "select content" button.
Verify you see login page of artemis and login using same instructor credentials.
Verify you see "LTI Single Configuration Test" course and select this course
Select multiple exercises where graded column is yes and no and click on "link" button
Verify you see selected exercises names and graded information on navigated Moodle page (Please note that if exercise is a graded exercise you will see maximum graded value as 100 - this is the intended behavior to match with current results service)
Click on add and verify the selected exercises are added to the Moodle course.

Testserver States

[!NOTE]
These badges show the state of the test servers.
Green = Currently available, Red = Currently locked

Review Progress

Performance Review

[ ] I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
[ ] I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

[ ] Code Review 1
[ ] Code Review 2

Manual Tests

[x] Test 1
[x] Test 2

Test Coverage

Client

Class/File
Line Coverage
Confirmation (assert/expect)

lti13-deep-linking.component.ts
90.62%
✅

Server

Class/File
Line Coverage
Confirmation (assert/expect)

LtiDeepLinkingService.java
97%
✅

LtiResource.java
100%
✅

Screenshots

Manually tested on TS11_Legacy Overall, works as expected.
But I was not redirected to the "editing external tool page" after clicking "Select add an activity or resourse" I needed to selecte "External Tool" manually from the listed options. Was it just missing in the testing description?

Thanks @xHadie I missed that step on the testing description, it is updated now.


Fix: explorer title is not selectable anymore #778

Fix issue that was introduced in 0e85c75 (#232)

Ping ping ping! :-)

@captbaritone EZ fix :-)

Thanks a lot, @kayhadrin! We are also facing this problem and can confirm that this PR solves it.
ping @AGS- and @asiandrummer

Alright, let's get this in - feels like a safe PR. :+1:


[WIP] Make @lower_castand @lower_constant return new references

Implements #4494 mainly using implicit casts. The new behaviour can be turned off by setting NUMBA_CAST_RETURNS_NEW_REFS=0 and NUMBA_LOWER_CONST_RETURNS_NEW_REFS=0 which should behave almost exactly like master except for the odd trivial cast and slightly different Numba IR for yield statements.
This PR

Introduces config switches NUMBA_CAST_RETURNS_NEW_REFS and NUMBA_LOWER_CONST_RETURNS_NEW_REFS to control the casting/lower constant behaviour
adds a new ref_type argument to @lower_cast and @lower_constant to let implementations declare which convention they are following (like in @iternext_impl) and adapts to the convention set in the config in the wrapper
adds clean-up for the references created if NUMBA_CAST_RETURNS_NEW_REFS=1 or NUMBA_LOWER_CONST_RETURNS_NEW_REFS=1
Making casts for yield values explicit (like it is already the case for return values)

Previous versions of this PR were favouring explicit casts (see). However, it turned out that making casts explicit requires Numba IR in SSA form and some changes to the typer. This in turn breaks many rewrites like parfors etc. While the previous incarnations of his PR worked for our internal code they never passed the Numba testsuite. Since porting #4445 to 0.48 looked like a significant effort we decided to go with implicit casts for now. This is also minimizing the impact on Numba IR rewrites.

Introduced separate constants LOWER_CONSTANT_RETURNS_NEW_REFS and CAST_RETURNS_NEW_REFS  as I realized changing the refcounting conventions for @lower_constant is much less effort than changing the conventions for @lower_cast. Hence it makes sense to implement them in separate PRs.

@sklam included your review comments. Still got the switchable version around in a local branch in case this will help making the transition.


Documentation: Installing on Windows

Environment
OS: Windows 7
Erlang/OTP: 18.3
EMQ: 2.0.6
Description
Link: http://emqtt.io/docs/v2/install.html#installing-on-windows
What means the sentence:
Warning
Cannot register EMQ-2.0 as a windows service.

Is a solution planned for the problem?

Did you try doing it with Admin privileges?
Alexander Tishchenko
mob: +7 (916) 704-61-57
<EMAIL_ADDRESS>
18 янв. 2017 г., в 17:35, Michael<EMAIL_ADDRESS>написал(а):
Environment
OS: Windows 7
Erlang/OTP: 18.3
EMQ: 2.0.6
Description
Link: http://emqtt.io/docs/v2/install.html#installing-on-windows
What means the senetence:
Warning
Cannot register EMQ-2.0 as a windows service.
Is a solution planned for the problem?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/emqtt/emqttd/issues/871, or mute the thread https://github.com/notifications/unsubscribe-auth/ASj_3_N37T8K1qZTg4OYib4o0rZ9xiy_ks5rTiM3gaJpZM4Lm8_B.

@michaelcplusplus @altishchenko The EMQ-2.0 cannot be registered as a window service. For we have to translate the etc/emq.conf to data/configs/app.config before the Erlang VM boot, we cannot use the erlsrv to register in bin/emqttd :(

I have no problem to install the service.
But I'm not able to start the service.
Messages from the Windows Eventlog:
ErlSrv:  emqttd_2.0.6: Not enough parameters for erlang service.
ErlSrv: emqttd_2.0.6: Could not start erlang machine
Registry entry:

@michaelcplusplus Would you like to check the bin/emqttd.cmd batch file and help fix this issue? Thanks.

Yes, of course.
The bin/emqttd.cmd file exists.
I hav eno problems to start the emqttd broker with one of the following commands from the command line:
bin\emqttd console
bin\emqttd start
The stopping works also from the command
bin\emqttd start
Best regards
Michael

I don't know if we could register the bin\emqttd start as a window service like the init.d on Linux.

Please note:
The registration works but the windows service could not be started.

The emqttd version 1.1.3 could be started as windows service. This version works fine for me.

I guess the missing starts_erl.cmd script is responsible for the misconduct.

Tested with emqttd-windows10-v2.1.0-rc.2 on windows 10.

The service can be installed.
But the service could not be started.
Windows Event viewer:
Entry 1

Entry 2

Tested with emqttd-windows10-v2.2-beta.2 on windows 10.
The service can be installed.
But the service could not be started.
Windows Event viewer:
Entry 1

Please see https://github.com/emqtt/emqttd/issues/803

Any solution for this problem, the latest versions are useless. I can't start service. Solution #803 also don't work on my PC.


Ad UI 6478/remove bg color utility

Proposed Changes
:bear: It's a big one...
I removed the util for bg-{$color} and all the dead classes that were used in the component and/or the example for the demo.
In some examples, I added hard coded colors to keep the visual effect (spacebox and Whitespace in Style - ColorBar, progress bar, members, etc).
The DatePicker needs love and i've found issues with it event in the demo from master. This needs to be it's own Jira.
Like this --->

I think the ColorBar should be also rework to use color in another way than  <ColorBar widthPerColor={{'#1372ec': 100}} />
I did not succeed to put color back in the collapsible header also :disappointed:
Potential Breaking Changes
Well, the date picker is the one bothering me the most. It's not really breaking, but this not looking great :sweat_smile:
Acceptance Criteria

[ ] The proposed changes are covered by unit tests
[ ] The potential breaking changes are clearly identified
[ ] README.md is adjusted to reflect the proposed changes (if relevant)

The error toast has no background color

That's a weird one, PO created the variable for status-color a while back but forgot to add the red one? :eyes: ....
But at the same time, they are not the same color than the critical-XXX, warning-XXX and success-XXX ... So I think I will pick the colors of the puces here for now: https://www.figma.com/file/A0wfZiuNFuDRp7zN3KOyA2/Platform-Light-Reskin?node-id=253%3A9662


Bugfix: null and undefined values fail

Thanks for that! It's merged and deployed to npm as 2.1.1.


add iceberg in pom

only need runtime jars, but it's better to add in pom. User can replace jars in spark home, don't need to change version in pom.

iceberg runtime and hive runtime, if s3 or oss, needs https://iceberg.apache.org/docs/latest/aws/ aws


Images not specified in .opf can't be loaded

But a small tweek to this function can resolve this issue.

The example file:
Charlotte's Web.zip

There are a couple of places where it tries to be lax, but I kind of feel that this is where one should draw the line, as it's a pretty fundamental requirement of the EPUB format that all resources must be listed in the manifest.

According to, https://github.com/w3c/epub-specs/issues/810, "iBooks, ADE, and Calibre" do allow this. So I guess we should, too. It still feels wrong, though...