samples/net/sockets/echo: Failed to connect to samples/net/echo_server for IPV6 test

I was trying to test echo_server along with echo_client with two different frdm_k64f boards.
The steps I have followed for server & client is as below:
Client :: zephyr/samples/net/echo_client
Server :: zephyr/samples/net/sockets/echo
cd zephyr/samples/net/echo_client
mkdir build && cd build && cmake -GNinja -DBOARD=frdm_k64f ../
ninja
cp -vr outdir/frdm_k64f/zephyr.bin /run/media/MBED/
cd zephyr/samples/net/sockets/echo
mkdir build && cd build && cmake -GNinja -DBOARD=frdm_k64f ../
ninja
cp -vr outdir/frdm_k64f/zephyr.bin /run/media/MBED1/
•Here I am trying make one frdm_k64f as client & other one as server.
The configuration used for building the server
For Freedom board with ethernet support
General config
CONFIG_NEWLIB_LIBC=y
Networking config
CONFIG_NETWORKING=y
CONFIG_NET_IPV4=n
CONFIG_NET_IPV6=y
CONFIG_NET_TCP=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y
Network driver config
CONFIG_TEST_RANDOM_GENERATOR=y
Network address config
CONFIG_NET_APP_SETTINGS=y
CONFIG_NET_APP_NEED_IPV4=n
CONFIG_NET_APP_MY_IPV4_ADDR="<IP_ADDRESS>"
CONFIG_NET_APP_PEER_IPV4_ADDR="<IP_ADDRESS>"
CONFIG_NET_APP_MY_IPV6_ADDR="2001:db8::1"
Network debug config
#CONFIG_NET_DEBUG_SOCKETS=y
CONFIG_SYS_LOG_NET_LEVEL=2
CONFIG_NET_APP_SERVER=y
The configuration used for building the echo_client
For Freedom board with ethernet support
For Freedom board with ethernet support
CONFIG_NETWORKING=y
CONFIG_NET_LOG=y
CONFIG_NET_IPV6=y
CONFIG_NET_IPV4=n
CONFIG_NET_DHCPV4=n
CONFIG_NET_UDP=n
CONFIG_NET_TCP=y
CONFIG_NET_STATISTICS=y
CONFIG_NET_PKT_RX_COUNT=50
CONFIG_NET_PKT_TX_COUNT=50
CONFIG_NET_BUF_RX_COUNT=50
CONFIG_NET_BUF_TX_COUNT=50
CONFIG_NET_IF_UNICAST_IPV6_ADDR_COUNT=3
CONFIG_NET_IF_MCAST_IPV6_ADDR_COUNT=5
CONFIG_NET_IF_UNICAST_IPV4_ADDR_COUNT=1
CONFIG_NET_MAX_CONTEXTS=10
CONFIG_INIT_STACKS=y
CONFIG_SYS_LOG_SHOW_COLOR=y
CONFIG_PRINTK=y
CONFIG_NET_L2_ETHERNET=y
CONFIG_NET_SHELL=y
CONFIG_NET_APP=y
CONFIG_NET_APP_NEED_IPV6=n
CONFIG_NET_APP_NEED_IPV4=y
CONFIG_NET_APP_CLIENT=y
CONFIG_NET_APP_SETTINGS=y
CONFIG_NET_APP_MY_IPV4_ADDR="<IP_ADDRESS>"
CONFIG_NET_APP_PEER_IPV4_ADDR="<IP_ADDRESS>"
CONFIG_NET_APP_MY_IPV6_ADDR="2001:db8::2"
CONFIG_NET_APP_PEER_IPV6_ADDR="2001:db8::1"
Logs on echo_server
BOOTING ZEP1.0-rc3 - BU 2018 08:23:Single-threaded TCP echo server waits for a connection on port 4242...
***** BUS FAULT *****
Executing thread ID (thread): 0x20002444
Faulting instruction address:  0xcd42
Precise data bus error
Address: 0x2d2100
Fatal fault in essential thread! Spinning...
Logs on echo_client
[dev/eth_mcux] [DBG] eth_0_init: MAC 00:04:9f:1c:84:b6
***** BOOTING ZEPHYR OS v1.11.0-rc3 - BUILD: Mar 12 2018 08:23:12 *****
Zephyr Shell, Zephyr version: 1.11.0-rc3
Type 'help' for a list of available commands
shell> [dev/eth_mcux] [INF] eth_mcux_phy_event: Enabled 100M full-duplex mode.
[echo-client] [INF] init_app: Run echo client
[echo-client] [ERR] connect_tcp: Cannot connect TCP (-60)
[echo-client] [ERR] start_tcp: Cannot init IPv6 TCP client (-60)
[echo-client] [ERR] connect_tcp: Cannot connect TCP (-60)
[echo-client] [ERR] start_tcp: Cannot init IPv4 TCP client (-60)
[echo-client] [ERR] send_tcp_data: Cannot send IPv6 data to peer (-57)
[echo-client] [ERR] send_tcp_data: Cannot send IPv4 data to peer (-57)
[echo-client] [INF] main: Stopping...
I am not able to find the communication between client & server.

samples/net/sockets/echo is binding to only IPv4 address. Running IPv6 client against it doesn't work. So closing issue.


fix: ci failed

go 1.14 does not include net.ErrClosed, so I upgraded it to 1.18.
In addition, add benchmark test in ci, use -count 1 for disable cache.

@roseduan Please take a look, thanks.

1.18 is a little high, I changed it to 1.16, also works.

1.18 is a little high, I changed it to 1.16, also works.

Yes，just need to be higher than version 1.14


Cannot open file "./results/olc/pre-flye-input-contigs.fa"

Hi,
I am experiencing some issues when trying to run athena-meta, more specifically in the OLC assembly step.. This is the error code I get in the log file:
2018-02-01 12:59:19 - --starting logging AssembleOLCStep --
2018-02-01 12:59:19 - jointly assemble bins with OLC
2018-02-01 12:59:19 - merge input contigs
2018-02-01 12:59:19 - ========== Exception ==========
2018-02-01 12:59:19 - Traceback (most recent call last):
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-01 12:59:19 -     chunk.run()
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 96, in run
2018-02-01 12:59:19 -     premergedfiltfa_path,
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 143, in filter_inputs
2018-02-01 12:59:19 -     ctg_size_map = util.get_fasta_sizes(mergedfa_path)
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/mlib/util.py", line 151, in get_fasta_sizes
2018-02-01 12:59:19 -     fasta = pysam.FastaFile(fa_path)
2018-02-01 12:59:19 -   File "pysam/libcfaidx.pyx", line 119, in pysam.libcfaidx.FastaFile.__cinit__
2018-02-01 12:59:19 -   File "pysam/libcfaidx.pyx", line 160, in pysam.libcfaidx.FastaFile._open
2018-02-01 12:59:19 - IOError: could not open file `./results/olc/pre-flye-input-contigs.fa`
2018-02-01 12:59:19 -
2018-02-01 12:59:19 - could not open file `./results/olc/pre-flye-input-contigs.fa`

Hope you can help me,
Jennifer

Hi Jennifer,
Sorry to hear you are experiencing some issues.  I recently updated to use Flye as the overlap assembler instead of canu and it’s quite possible I made some oversights in the transition.  A couple of questions to help me figure it out:

what version are you running?  Is it the tagged release and if not, git commit tag?
did you rerun it on an existing run after updating your source repository?  If so, I would remove <work_dir>/results/olc to remove any stale state from the previous version and retry.
did you get the small (newly referenced) l. gasseri dataset referenced in the README to run?  Either in the docker or on your own set up.

Please let me know when you have the chance and we can figure out the issue.
Thanks!
alex
On Feb 1, 2018, at 4:20 AM, jennifertheland <notifications<EMAIL_ADDRESS>wrote:
Hi,
I am experiencing some issues when trying to run athena-meta, more specifically in the OLC assembly step.. This is the error code I get in the log file:
2018-02-01 12:59:19 - --starting logging AssembleOLCStep --
2018-02-01 12:59:19 - jointly assemble bins with OLC
2018-02-01 12:59:19 - merge input contigs
2018-02-01 12:59:19 - ========== Exception ==========
2018-02-01 12:59:19 - Traceback (most recent call last):
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-01 12:59:19 -     chunk.run()
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 96, in run
2018-02-01 12:59:19 -     premergedfiltfa_path,
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 143, in filter_inputs
2018-02-01 12:59:19 -     ctg_size_map = util.get_fasta_sizes(mergedfa_path)
2018-02-01 12:59:19 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/mlib/util.py", line 151, in get_fasta_sizes
2018-02-01 12:59:19 -     fasta = pysam.FastaFile(fa_path)
2018-02-01 12:59:19 -   File "pysam/libcfaidx.pyx", line 119, in pysam.libcfaidx.FastaFile.cinit
2018-02-01 12:59:19 -   File "pysam/libcfaidx.pyx", line 160, in pysam.libcfaidx.FastaFile._open
2018-02-01 12:59:19 - IOError: could not open file ./results/olc/pre-flye-input-contigs.fa
2018-02-01 12:59:19 -
2018-02-01 12:59:19 - could not open file ./results/olc/pre-flye-input-contigs.fa
Hope you can help me,
Jennifer
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/abishara/athena_meta/issues/8, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACzgIb7eppm5KnZEZstj-Jm-CAjrfcIzks5tQawAgaJpZM4R1hOZ.

Hello again,
I am running Flye version 2.3.1-release, I installed all programs on my server after you changed to Flye and everything is up to date according to git.
I just tried running athena-meta afresh on the dataset referenced in the README. I get the following output:
2018-02-02 09:18:21 - --starting logging AssembleMetaBinnedStep.bin.0 --
2018-02-02 09:18:21 - performing local assembly for 1 seeds
2018-02-02 09:18:21 - targeting 100x short-read subassembly coverage
2018-02-02 09:18:21 - using barcodes mapped within 10000bp from seed end-points for seed subassembly
2018-02-02 09:18:21 - assembling barcoded reads for seed NODE_48_length_1737_cov_9.13912
2018-02-02 09:18:21 - determing local assemblies
2018-02-02 09:18:21 - 2 initial link candidates to check
2018-02-02 09:18:21 -   - 2 pass reciprocal filtering
2018-02-02 09:18:21 - root-ctg:NODE_48_length_1737_cov_9.13912;numreads:237;checks:2;trunc-checks:False;asms:2;trunc-asms:False
2018-02-02 09:18:21 -   - found 3 candidates
2018-02-02 09:18:21 - performing local assemblies
2018-02-02 09:18:21 - assembling with neighbor NODE_60_length_345_cov_8.40345
2018-02-02 09:18:21 -   - 23 orig barcodes
2018-02-02 09:18:21 -   - 23 downsampled barcodes
2018-02-02 09:18:21 -   - 5.46474181011x estimated local coverage
2018-02-02 09:18:21 -   - 2 min_support required
2018-02-02 09:18:21 - assembly failed to produce contig.fa
2018-02-02 09:18:21 - ========== Exception ==========
2018-02-02 09:18:21 - Traceback (most recent call last):
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-02 09:18:21 -     chunk.run()
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_meta_bins.py", line 82, in run
2018-02-02 09:18:21 -     self.do_local_assembly(ctg, asmdir)
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_meta_bins.py", line 147, in do_local_assembly
2018-02-02 09:18:21 -     local_asm_results = asm.assemble(local_asms, filt_ctgs=seed_ctgs)
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/assembler_tools/barcode_assembler/local.py", line 83, in assemble
2018-02-02 09:18:21 -     contig_path = self._do_idba_assembly(local_asm)
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/assembler_tools/barcode_assembler/local.py", line 173, in _do_idba_assembly
2018-02-02 09:18:21 -     raise Exception()
2018-02-02 09:18:21 - Exception
2018-02-02 09:18:21 -
2018-02-02 09:18:21 -

I also tried to run my own dataset one more time from the top and I now get the same error code as I get with the testset (i.e. the one above). I had processed my reads a bit different from you, but I have changed them to match yours now...
Jennifer

Hi Jennifer,
Okay, thanks for sharing some more information.  I think I know what might be happening.  Can you do ‘which idba_subasm’ and let me know if that yields a valid path to a binary in your install environment?  I have also since updated Athena to have a new external dependency on a version of idba that I modified:
https://github.com/abishara/idba/releases/tag/1.1.3a1
previously we used one that did not have these changes.  If idba_subasm is not visible in your path, can you install the version specified above so that it is and retry?  Sorry that this update was not made more explicit in the README.  I will go ahead and make this much more explicit and also add some error checking if it turns out to be an issue.  The updated docker container may also be helpful with diagnosing possible issues with setup.
Thanks!
alex
On Feb 2, 2018, at 1:49 AM, jennifertheland <notifications<EMAIL_ADDRESS>wrote:
Hello again,
I am running Flye version 2.3.1-release, I installed all programs on my server after you changed to Flye and everything is up to date according to git.
I just tried running athena-meta afresh on the dataset referenced in the README. I get the following output:
2018-02-02 09:18:21 - --starting logging AssembleMetaBinnedStep.bin.0 --
2018-02-02 09:18:21 - performing local assembly for 1 seeds
2018-02-02 09:18:21 - targeting 100x short-read subassembly coverage
2018-02-02 09:18:21 - using barcodes mapped within 10000bp from seed end-points for seed subassembly
2018-02-02 09:18:21 - assembling barcoded reads for seed NODE_48_length_1737_cov_9.13912
2018-02-02 09:18:21 - determing local assemblies
2018-02-02 09:18:21 - 2 initial link candidates to check
2018-02-02 09:18:21 -   - 2 pass reciprocal filtering
2018-02-02 09:18:21 - root-ctg:NODE_48_length_1737_cov_9.13912;numreads:237;checks:2;trunc-checks:False;asms:2;trunc-asms:False
2018-02-02 09:18:21 -   - found 3 candidates
2018-02-02 09:18:21 - performing local assemblies
2018-02-02 09:18:21 - assembling with neighbor NODE_60_length_345_cov_8.40345
2018-02-02 09:18:21 -   - 23 orig barcodes
2018-02-02 09:18:21 -   - 23 downsampled barcodes
2018-02-02 09:18:21 -   - 5.46474181011x estimated local coverage
2018-02-02 09:18:21 -   - 2 min_support required
2018-02-02 09:18:21 - assembly failed to produce contig.fa
2018-02-02 09:18:21 - ========== Exception ==========
2018-02-02 09:18:21 - Traceback (most recent call last):
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-02 09:18:21 -     chunk.run()
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_meta_bins.py", line 82, in run
2018-02-02 09:18:21 -     self.do_local_assembly(ctg, asmdir)
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_meta_bins.py", line 147, in do_local_assembly
2018-02-02 09:18:21 -     local_asm_results = asm.assemble(local_asms, filt_ctgs=seed_ctgs)
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/assembler_tools/barcode_assembler/local.py", line 83, in assemble
2018-02-02 09:18:21 -     contig_path = self._do_idba_assembly(local_asm)
2018-02-02 09:18:21 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/assembler_tools/barcode_assembler/local.py", line 173, in _do_idba_assembly
2018-02-02 09:18:21 -     raise Exception()
2018-02-02 09:18:21 - Exception
2018-02-02 09:18:21 -
2018-02-02 09:18:21 -
I also tried to run my own dataset one more time from the top and I now get the same error code as I get with the testset (i.e. the one above). I had processed my reads a bit different from you, but I have changed them to match yours now...
Jennifer
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/abishara/athena_meta/issues/8#issuecomment-362538465, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACzgIcssETuG2W6bK09Xb-pc-7dGFSuoks5tQto2gaJpZM4R1hOZ.

Hi,
So idba was the problem and I managed to get athena running. However, I stumble upon some new issues when trying to launch flye. I get the following error message:
launching Flye OLC assembly
cmd flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000
[2018-02-05 12:36:27] INFO: Running Flye 2.3.1-release
[2018-02-05 12:36:27] INFO: Assembling reads
[2018-02-05 12:36:27] INFO: Reading sequences
[2018-02-05 12:36:27] WARNING: Estimated read coverage is 3, the assembly is not guaranteed to be optimal in this setting. Are you sure that the genome size was entered correctly?
[2018-02-05 12:36:27] INFO: Generating solid k-mer index
[2018-02-05 12:36:45] INFO: Counting kmers (1/2):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:45] INFO: Counting kmers (2/2):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:47] INFO: Filling index table
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:49] INFO: Extending reads
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:51] INFO: Added 1057 singleton reads
[2018-02-05 12:36:51] INFO: Assembled 1057 draft contigs
[2018-02-05 12:36:51] INFO: Generating contig sequences
[2018-02-05 12:36:51] INFO: Performing repeat analysis
[2018-02-05 12:36:51] INFO: Reading sequences
[2018-02-05 12:36:51] INFO: Building repeat graph
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:37:12] INFO: Aligning reads to the graph
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:37:33] ERROR: Segmentation fault! Backtrace:
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x41a0ae]
[2018-02-05 12:37:33] ERROR:    /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f9a8754b4b0]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x42665d]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x4242c9]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x404f8c]
[2018-02-05 12:37:33] ERROR:    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f9a87536830]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x405f41]
[2018-02-05 12:37:33] ERROR: Command '['flye-repeat', '-k', '15', '-l', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/flye.log', '-t', '4', '-v', '1000', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/0-assembly/draft_assembly.fasta', './results/olc/flye-input-contigs.fa', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/2-repeat', '/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/flye/resource/asm_corrected_reads.cfg']' returned non-zero exit status 1
2018-02-05 12:37:33 - ========== Exception ==========
2018-02-05 12:37:33 - Traceback (most recent call last):
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-05 12:37:33 -     chunk.run()
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 128, in run
2018-02-05 12:37:33 -     subprocess.check_call(cmd, shell=True)
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/subprocess.py", line 540, in check_call
2018-02-05 12:37:33 -     raise CalledProcessError(retcode, cmd)
2018-02-05 12:37:33 - CalledProcessError: Command 'flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000' returned non-zero exit status 1
2018-02-05 12:37:33 -
2018-02-05 12:37:33 - Command 'flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000' returned non-zero exit status 1
Exception in thread Thread-6:
Traceback (most recent call last):
  File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/multiprocessing/pool.py", line 389, in _handle_results
    task = get()
TypeError: ('__init__() takes at least 3 arguments (1 given)', <class 'subprocess.CalledProcessError'>, ())

/Jennifer

Hi Jennifer,
Glad to hear that took care of your first issue.  I actually have not yet encountered the segmentation fault in the Flye overlap assembly stage before.  Would you mind sharing your input reads stored in the run directory:
/results/olc/flye-input-contigs.fa
This can be quite large (~Gb) so you might want to share through google drive, box, s3, etc if you are willing.  Otherwise this will be very difficult for me to diagnose as it isn’t something I have encountered with our runs on a relatively diverse samples.  Additionally can I ask how deep is your sequencing run (how many total Gb) and what is the nature of the sample (gut, synthetic mixture, soil, etc).  This would also give me some idea.  From what is specified to —genome-size in the command below implies your sample is of pretty low complexity.
Thanks for your patience and hopefully we can resolve this soon!
alex
On Feb 5, 2018, at 3:59 AM, jennifertheland <notifications<EMAIL_ADDRESS>wrote:
Hi,
So idba was the problem and I managed to get athena running. However, I stumble upon some new issues when trying to launch flye. I get the following error message:
launching Flye OLC assembly
cmd flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000
[2018-02-05 12:36:27] INFO: Running Flye 2.3.1-release
[2018-02-05 12:36:27] INFO: Assembling reads
[2018-02-05 12:36:27] INFO: Reading sequences
[2018-02-05 12:36:27] WARNING: Estimated read coverage is 3, the assembly is not guaranteed to be optimal in this setting. Are you sure that the genome size was entered correctly?
[2018-02-05 12:36:27] INFO: Generating solid k-mer index
[2018-02-05 12:36:45] INFO: Counting kmers (1/2):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:45] INFO: Counting kmers (2/2):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:47] INFO: Filling index table
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:49] INFO: Extending reads
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:51] INFO: Added 1057 singleton reads
[2018-02-05 12:36:51] INFO: Assembled 1057 draft contigs
[2018-02-05 12:36:51] INFO: Generating contig sequences
[2018-02-05 12:36:51] INFO: Performing repeat analysis
[2018-02-05 12:36:51] INFO: Reading sequences
[2018-02-05 12:36:51] INFO: Building repeat graph
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:37:12] INFO: Aligning reads to the graph
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:37:33] ERROR: Segmentation fault! Backtrace:
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x41a0ae]
[2018-02-05 12:37:33] ERROR:    /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f9a8754b4b0]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x42665d]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x4242c9]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x404f8c]
[2018-02-05 12:37:33] ERROR:    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f9a87536830]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x405f41]
[2018-02-05 12:37:33] ERROR: Command '['flye-repeat', '-k', '15', '-l', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/flye.log', '-t', '4', '-v', '1000', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/0-assembly/draft_assembly.fasta', './results/olc/flye-input-contigs.fa', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/2-repeat', '/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/flye/resource/asm_corrected_reads.cfg']' returned non-zero exit status 1
2018-02-05 12:37:33 - ========== Exception ==========
2018-02-05 12:37:33 - Traceback (most recent call last):
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-05 12:37:33 -     chunk.run()
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 128, in run
2018-02-05 12:37:33 -     subprocess.check_call(cmd, shell=True)
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/subprocess.py", line 540, in check_call
2018-02-05 12:37:33 -     raise CalledProcessError(retcode, cmd)
2018-02-05 12:37:33 - CalledProcessError: Command 'flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000' returned non-zero exit status 1
2018-02-05 12:37:33 -
2018-02-05 12:37:33 - Command 'flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000' returned non-zero exit status 1
Exception in thread Thread-6:
Traceback (most recent call last):
File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/multiprocessing/pool.py", line 389, in _handle_results
task = get()
TypeError: ('init() takes at least 3 arguments (1 given)', <class 'subprocess.CalledProcessError'>, ())
/Jennifer
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/abishara/athena_meta/issues/8#issuecomment-363064461, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACzgId3XDQLXmdsNVnLF2onEr-ugoBqcks5tRu0GgaJpZM4R1hOZ.

Hi Jennifer,
I wanted to check in to see if I could help with issues you may still be having.
I took a closer look at the log you sent me and noticed in the path it says “test_data_sub”.  It looks like you may have subsetted the data in the *bam, but used the full output seed assembly from the initial metaspades, which looks like a single ~4Mb organism.  I asked the developer who maintains the overlap assembler Flye, in which it is currently segfaulting.  He mentioned that the perceived low coverage from subsetting the data like that may cause a segfault when building the repeat graph with the tagged version currently used in Athena.  If you give me a little more info and share some of the inputs, I think we should be able to figure it out.
Let me know if you’d still like some help.
Thanks!
alex
On Feb 5, 2018, at 9:28 AM, Alex Bishara <abishara<EMAIL_ADDRESS>wrote:
Hi Jennifer,
Glad to hear that took care of your first issue.  I actually have not yet encountered the segmentation fault in the Flye overlap assembly stage before.  Would you mind sharing your input reads stored in the run directory:
/results/olc/flye-input-contigs.fa
This can be quite large (~Gb) so you might want to share through google drive, box, s3, etc if you are willing.  Otherwise this will be very difficult for me to diagnose as it isn’t something I have encountered with our runs on a relatively diverse samples.  Additionally can I ask how deep is your sequencing run (how many total Gb) and what is the nature of the sample (gut, synthetic mixture, soil, etc).  This would also give me some idea.  From what is specified to —genome-size in the command below implies your sample is of pretty low complexity.
Thanks for your patience and hopefully we can resolve this soon!
alex
On Feb 5, 2018, at 3:59 AM, jennifertheland <notifications<EMAIL_ADDRESS>wrote:
Hi,
So idba was the problem and I managed to get athena running. However, I stumble upon some new issues when trying to launch flye. I get the following error message:
launching Flye OLC assembly
cmd flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000
[2018-02-05 12:36:27] INFO: Running Flye 2.3.1-release
[2018-02-05 12:36:27] INFO: Assembling reads
[2018-02-05 12:36:27] INFO: Reading sequences
[2018-02-05 12:36:27] WARNING: Estimated read coverage is 3, the assembly is not guaranteed to be optimal in this setting. Are you sure that the genome size was entered correctly?
[2018-02-05 12:36:27] INFO: Generating solid k-mer index
[2018-02-05 12:36:45] INFO: Counting kmers (1/2):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:45] INFO: Counting kmers (2/2):
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:47] INFO: Filling index table
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:49] INFO: Extending reads
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:36:51] INFO: Added 1057 singleton reads
[2018-02-05 12:36:51] INFO: Assembled 1057 draft contigs
[2018-02-05 12:36:51] INFO: Generating contig sequences
[2018-02-05 12:36:51] INFO: Performing repeat analysis
[2018-02-05 12:36:51] INFO: Reading sequences
[2018-02-05 12:36:51] INFO: Building repeat graph
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:37:12] INFO: Aligning reads to the graph
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
[2018-02-05 12:37:33] ERROR: Segmentation fault! Backtrace:
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x41a0ae]
[2018-02-05 12:37:33] ERROR:    /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f9a8754b4b0]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x42665d]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x4242c9]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x404f8c]
[2018-02-05 12:37:33] ERROR:    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f9a87536830]
[2018-02-05 12:37:33] ERROR:    flye-repeat() [0x405f41]
[2018-02-05 12:37:33] ERROR: Command '['flye-repeat', '-k', '15', '-l', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/flye.log', '-t', '4', '-v', '1000', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/0-assembly/draft_assembly.fasta', './results/olc/flye-input-contigs.fa', '/home/jennifer/test_data_sub/results/olc/flye-asm-1/2-repeat', '/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/flye/resource/asm_corrected_reads.cfg']' returned non-zero exit status 1
2018-02-05 12:37:33 - ========== Exception ==========
2018-02-05 12:37:33 - Traceback (most recent call last):
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/pipeline.py", line 50, in _run_chunk
2018-02-05 12:37:33 -     chunk.run()
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/site-packages/athena/stages/assemble_olc.py", line 128, in run
2018-02-05 12:37:33 -     subprocess.check_call(cmd, shell=True)
2018-02-05 12:37:33 -   File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/subprocess.py", line 540, in check_call
2018-02-05 12:37:33 -     raise CalledProcessError(retcode, cmd)
2018-02-05 12:37:33 - CalledProcessError: Command 'flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000' returned non-zero exit status 1
2018-02-05 12:37:33 -
2018-02-05 12:37:33 - Command 'flye --subassemblies ./results/olc/flye-input-contigs.fa --out-dir ./results/olc/flye-asm-1 --genome-size 4079237 --threads 4 --min-overlap 1000' returned non-zero exit status 1
Exception in thread Thread-6:
Traceback (most recent call last):
File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/shared/external/miniconda3/envs/athena_assembly/lib/python2.7/multiprocessing/pool.py", line 389, in _handle_results
task = get()
TypeError: ('init() takes at least 3 arguments (1 given)', <class 'subprocess.CalledProcessError'>, ())
/Jennifer
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/abishara/athena_meta/issues/8#issuecomment-363064461, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACzgId3XDQLXmdsNVnLF2onEr-ugoBqcks5tRu0GgaJpZM4R1hOZ.


Automatic unique CONSTR_IDs

Is your feature request related to a problem? Please describe.
When trying to define Union'd PlutusData in OpShin/PyCardano, one has to take care that constructor ids are unique for all elements in the Union. While other languages like PlutusTx take care of this automatically (assigning all elements in a Union unique ids at definition) in PyCardano we have to manually pick a set of unique CONSTR_IDs. Also we want to be able to benefit from the free declarability of Unions in Python, which lets us group together types across definitions (i.e Union[A,B] and Union[A,C], this is actually not possible with any other Cardano SC language !)
Describe the solution you'd like
We definetly want a few properties on the CONSTR_IDs

small: ideally the constr_id integer should be as small as possible, as smaller integers are encoded more efficiently in CBOR and save the end user minutxo and txfees (constr_ids are encoded as the cbor tag up to 7 bit size, after that encoded as generic integer)
unique: There should be as little overlap with other values as possible, so that we can group together classes in unions without having to worry about setting/overwriting the constr id
overwritable: In order to mimic external interfaces such as the ledger, we need to be able to manually set the CONSTR_ID. We also may want to manually set the IDs in case of collisions
deterministic: Datatypes that are defined in libraries may be imported in arbitrary contexts. the constr_id must therefore not depend on i.e. what other Unions the datatype is being used in or what other datatypes are declared in its surroundings

Overwritability is an implementation detail that we simply need to bear in mind when adopting this change.
IMO the best solution is based on a hash of the class name. The main question is now how to process the hash to serve for  the goal of minimal size.
Alternative A: CONSTR_ID = hash(classname) % 127
This solution truncates the hash of the classname to the last 7 bits, giving us a constructor id between 0 and 127, neatly encodable in a single byte. However, due to the birthday paradox, this may result in a collision with another constructor id from 23 defined classes onwards. In practice we see ~5-10 user defined classes per contract so this could be fine if alternative B is too expensive
Alternative B: CONSTR_ID = hash(classname) % (2**32)
This solution truncates the hash of the classname to 32 bits, nicely fitting into a normal integer. This will reduce the likelyhood of conflicts to a negligible amount, however costing 3 bytes more than solution A per datum (mind that these datums end up on chain). It should be investigated how much the impact of this is on minutxo and script execution cost.
Describe alternatives you've considered
Solutions similar to auto() in enum (global incremental counter that increases with every definition of a plutusdatum) is not a good idea because it breaks determinism.
Additional context
OpShin lets you write smart contracts in Cardano and uses the PlutusData class for native data.
In haskell, data is usually defined together with all alternatives of the same type (i.e. A = B | C | D) while in python/Pycardano/opshin the classes are first defined, then later combined to unions (i.e. class B; class C; class D; A = Union[B, C, D])
Requesting comments from @cffls and @juliusfrost, but will most likely just go ahead and implement this as soon as I got time for it (~1-2 weeks from now)

Thanks for the great proposal! I like the simplicity in the hashing solution. However, I am not sure if hashing is entirely deterministic. I could be wrong, but will determinism be broken if classes run into hashing collision?

I could be wrong, but will determinism be broken if classes run into hashing collision?

Good point.  However no, in case of hash collisions, we would require the user to manually set a CONSTR_ID for the class

There seems to be a difference in serialization size even for CONSTR_ID under 127:
>>> a = CBORTag(127, [])
>>> b = CBORTag(0, [])
>>> len(dumps(a))
3
>>> len(dumps(b))
2

I am wondering if it is possible to automatically assign constructor id for each class based on their dependencies during serialization. It will require some type of inspection tool, e.g. pyreverse, to create a dependency graph. With the graph, I think it would be possible to auto assign CONSTR_ID semi-deterministically starting from 0, similar to what unstableMakeIsData does in PlutusTx.

What kind of dependencies do you mean? If you mean invocation in Union, I would discourage that since it may break when the same datum is being used in different contracts and contexts.

I meant the dependency of datums and contracts in the scope of a dApp. Not sure if it is a good idea to share the same datum class across difference dApps though.

Ultimately this will be needed for all kinds of aggregators and smart contract chains :)

Neat side effect of this change: You can check the constructor ID of almost arbitrary data on chain.
If your know all Plutus data definitions you can even retreive the semantic meaning. Otherwise you can still do more advanced things like the isinstance check on arbitrary builtin data as suggested in https://github.com/OpShin/opshin/issues/70


About principles on generating DRR

Hello! your work is brilliant! But I am confused about princples on on generating DRRs, could you tell me more about it? I remember a paper in itk website focused on it ,maybe published in 2010, but I cannot find it on itk website now.

DRR is the discretization of the x-ray transform, i.e., line integrals. There are several solutions to do it, e.g., https://aapm.onlinelibrary.wiley.com/doi/pdf/10.1118/1.595715 or, the implemented version in ITK, https://ieeexplore.ieee.org/document/4307572. I'm not aware of such an ITK article, maybe https://doi.org/10.54294/6f280b? It doesn't seem to contain an article... Maybe youvcan continue the discussion on discourse.itk.org, it does not seem to be a real issue in ITK.

Reference in

Thanks for replying，you have already solved the probem for me!

DRR is the discretization of the x-ray transform, i.e., line integrals. There are several solutions to do it, e.g., https://aapm.onlinelibrary.wiley.com/doi/pdf/10.1118/1.595715 or, the implemented version in ITK, https://ieeexplore.ieee.org/document/4307572. I'm not aware of such an ITK article, maybe https://doi.org/10.54294/6f280b? It doesn't seem to contain an article... Maybe youvcan continue the discussion on discourse.itk.org, it does not seem to be a real issue in ITK.

Thanks for replying，you have already solved the probem for me!

DRR is the discretization of the x-ray transform, i.e., line integrals. There are several solutions to do it, e.g., https://aapm.onlinelibrary.wiley.com/doi/pdf/10.1118/1.595715 or, the implemented version in ITK, https://ieeexplore.ieee.org/document/4307572. I'm not aware of such an ITK article, maybe https://doi.org/10.54294/6f280b? It doesn't seem to contain an article... Maybe youvcan continue the discussion on discourse.itk.org, it does not seem to be a real issue in ITK.

Hello！I want to verify if the example given bellow https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/Filtering/DigitallyReconstructedRadiograph1.cxx use joseph's method to generate drrs?

Thanks, I will read the code.


Use Uint8Array as internal repr of Bytes and fix #117

builder.c code JS Uint8Array -> C uint8_t * and vice versa works, ensured by a few passed tests (valueReturn: JS -> C, readRegister: C -> JS).
To discuss:

should we keep Bytes for backward-compatible. Have Bytes Uint8Array | String, or a class which constructor can take a string and check it's all in range, or just alias of Uint8Array, or just drop Bytes and use Uint8Array for low level APIs
Uint8Array <> String decode/encode. In nodejs, this is typically via Buffer (a subclass and enhanced version of Uint8Array) or TextEncoder/TextDecoder, but none of them is in quickjs. WIth basic String.charCodeAt/fromCharCode, I can make a simplest latin1 encoder/decoder (each Uint8 is mapped to the same 0-255 char code). With the unicode C library shipped with quickjs, it seems possible to expose a UTF-8 and UTF-16 text encoder/decoder. I think these together are sufficiently good. Other ideas are welcome.
We serialize state to JSON, this is problematic with unicode characters. After we have the correct enforcement on storage key / value to be Uint8Array, unicode characters will be correctly rejected, but we must come up a strategy in our sdk's auto-serialization to handle user object with unicode string properties. A few possible strategy:

Throw error at any unicode string character, only allow latin1 chars.
Always utf-8 decode at serialization, and utf-8 encode at deserialization
Use a schema-less binary format to replace JSON, for example, CBOR

@volovyks @austinabell What are your thoughts? Thank you!

From discussion with @volovyks , I propose the following design (all alternatives I can think of are in PR description):

We will change to type Bytes = Uint8Array | string.

Use string is unchecked and auto utf-8 decoded (same as current behavior, and same as browser/nodejs TextDecoder's default encoding).
Use string will be warned at compile time.

For those api.ts functions that were returning Bytes, we'll return Uint8Array. (Breaking change)
Provide TextEncoder, TextDecoder class, implemented in C, can do utf-8. utf-16le/be and latin1 encoding
Auto JSON serialization on arguments, state, collections are auto utf-8 decoded to uint8_t* in C, and auto utf-8 encoded when deserialize. (same as current behavior, but we'll explicitly document this fact)
Not in this PR, we can provide a CBOR serializer and borsh serializer support, that does not need TextDecoder.

In summary, impact to user is in low level APIs. Function returning Bytes now return Uint8Array. Function taking Bytes can still use string, but recommended to use Uint8Array. High level APIs (nearbindgen & collections) are unchanged. Given that most user contracts are built with high level APIs, the impact is minimum.
Please comment if there's anything looks wrong or there's a better design!

Why do you need to reject unicode characters if everything is being translated to utf8?

The problem is ambiguity. A utf-8 sequence in byte and a string that is utf-8 encoded from same bytes will be serialized to the same thing, and you cannot know which one it is from when doing deserialize.

Did you mean only allow utf8 chars? Why latin1? (curiosity) The changes feel like it's actually utf8

I want to note that this bullet point is one of the possible approaches, but not the approach implemented in this PR. So yeah your observation "The changes feel like it's actually utf8" is right. This alternative approach is to restrict to must pass a JS string full of latin1 character (char code 0-255 only) to ensure correctness in deserialization.

Hi all, what is the status on this PR?
In the current form, it's pretty much impossible to use bytes in a contract without the data being mangled in UTF-16 conversion issues, making any application which involves raw bytes unusable. As a workaround I am base64 encoding everything but this seems extremely inefficient.
IMO, bytes should = Uint8Array and nothing else. For example, near.ecrecover takes bytes as input. Why would you want the ambiguity of a signature with string.length == 64 being an unknown number bytes and an invalid signature?
I think this PR needs urgent attention as the lack of being able to using a raw bytes array makes the JS SDK unusable for many applications.

@no2chem you are right, I'll look into this week

superseded by #308 , basically what we agreed here https://github.com/near/near-sdk-js/pull/195#issuecomment-1231089443 + @no2chem suggested:

bytes should = Uint8Array and nothing else

are implemented. Let's review and further discuss in #308


ctest failures

Hello,
I've installed all dependencies which took a while and am now on the ctest step of alphafill.
When I run ctest --test-dir build the first test is successful then the next two (run_process and validation) fail.
The cause of failure of the second test is that AlphaFill can't seem to find or load the mmcif_pdbx dictionary. I've downloaded and installed mmcif_pdbx. Then I realized from googling that mmcif_pdbx.dic is actually found in libcifpp/rsrc and I confirmed this locally.
Any idea how I can direct alphafill to the dictionary or why it's not detecting it?
Specs:
Rock Linux 9.2
boost 1.78
cmake 3.30
gcc and gxx 11.3.1
make 4.4.1
mmcif-pdbx 2.0.1
libcifpp, libmcfp, and libzeep were cloned and installed today (9/17/24, so should be updated versions)
Thanks for any help,

Alphafill uses libcifpp. And libcifpp attempts to load the dictionary from a 'resource'. That is, if you use mrc, then this dictionary will be built in in the executable. If you do not use mrc, you will have to install libcifpp first and make sure this is done correctly. Alternatively, you can specify resource locations using an environment variable.
Please read this related documentation: https://www.hekkelman.net/libcifpp-doc/resources.html

I removed libcifpp and performed the installation as described on the libcifpp page. The only difference between this time and last time was using ccmake. Anyway, AlphaFill is now finding everything, is installed, and passing the tests.
Thanks!


client-side rendering client.js shortcut

So, if today you render a component doing something like:
...
<oc-component href="//registry.com/component/1.2.X/?hello=world"></oc-component>
<script src="//registry.com/oc-client/client.js"></script>

it would be super cool to have an equivalent single line like this:
<script src="//registry.com/component/1.2.X/client.js?hello=world"></script>

The response would be something like a document.write containing the oc-component tag + the IE8 polyfill + the oc-client.

I don't think this i required anymore. Closing it.


Issue with validating a plan generated by not integrated planner

Describe the bug
I generated a plan through a planner that is not supported by unified planning and then ported the actions of the plan using the following code:

reader = PDDLReader()
task = reader.parse_problem('domain.pddl', 'problem.pddl")

with Compiler(problem_kind=self.task.kind, compilation_kind=CompilationKind.GROUNDING) as grounder:
            ground_problem = grounder.compile(self.task, compilation_kind=CompilationKind.GROUNDING).problem

# used some external planner defined in the dict call model.

plan = []
## linearize partial-order plan
for step in range(self.encoder.horizon):
    for action in self.encoder.ground_problem.actions:
        if is_true(model[self.encoder.action_variables[step][action.name]]):
            plan.append(ActionInstance(action))
ufplan = SequentialPlan(plan, ground_problem)

After converting the extracted plan, I used:
with PlanValidator(problem_kind=task.kind, plan_kind= ufplan) as validator:
    if validator.validate(task, ufplan):
        print('The plan is valid')
    else:
        print('The plan is invalid')

I get an error saying that some action does not belong to the problem.
Expected behavior
I should receive a plan that is valid or not.
Desktop (please complete the following information):

OS: MacOS
Browser safari
Version 16.3

Hi @MFaisalZaki , there are 2 problems with your code:

the SequentialPlan second argument is the environment, so you should pass task.environment (or ignore it, the default should be OK)
you are creating a plan for the grounded problem and you are validating it for the original problem (task).

So, you either validate it for the grounded problem, or convert every ActionInstance of the plan into the equivalent action of the original problem.
How to do this? It is explained in the section 5.9 of the notebook on compilers
Let me know if you still have issues.

Hi @Framba-Luca,
Thanks for your reply. I have updated the plan validator to use the grounded task and I get this error:

Here is the updated code:
with PlanValidator(problem_kind=task.kind, plan_kind= ufplan) as validator:
    if validator.validate(ground_problem, ufplan):
        print('The plan is valid')
    else:
        print('The plan is invalid')

Here are the modifications you need in your code (with a comment explaining why):
with PlanValidator(
     problem_kind=ground_problem.kind, # You are getting a validator for the ground problem
     plan_kind= ufplan.kind, # You need to pass the kind of the plan, not the whole plan
) as validator:
    if validator.validate(ground_problem, ufplan):
        print('The plan is valid')
    else:
        print('The plan is invalid')

It works now. Thanks for your guidance, @Framba-Luca.


Update deployment

Options to be considered

Serverless streamlit - currently doesn't work on Firefox on Mac. Problem!
Quarto with Shinylive - possible, should consider if stuff gets more complex, but overhead time to rewrite (good project though if just in general want to get more familiar with Quarto and/or Shiny for Python)
Web host - probably will do this, just need to look up/figure out. $

On a web host now, instructions here


Problem adding custom robot model

There seems to be a problem with importing custom robot models. I started by adding my robot model, formatted as was described in the online documentation, to the folder that "help"/"Show FRC Data Folder" sent me to. After restarting AdvantageScope and my PC (Windows), the robot model did not show up.
Thinking that the issue could have easily been on my part, I downloaded and unzipped the Robot_BananaSplitV1 model and added it to the frcData folder as well. After restarting AdvantageScope again, neither robot showed up in the 3D field robot dropdown.
Looking through past issues, I saw one  from earlier this year. Because this seemed to work on January 20th, I backdated my version of AdvantageScope to v2.2.0 and had the same issue.
Looking through the code, I saw that the "frcData" folder was under a constant called LEGACY_ASSETS, so I created a new folder within AppData/Roaming/AdvantageScope called "userAssets" to see if it would fix the issue. It did not. I believe that I tested adding this folder in the most recent release as well, to no avail.
Am I missing something?

The 2024 version of AdvantageScope will use a new format for custom assets, but it's still under active development. The online documentation linked in the repo is for the current development version. For adding custom models, you should reference the documentation built-in to the latest stable release (v2.3.0) by clicking the 📖 icon. The older docs are also online here.
With the 2023 format, everything is under "frcData" with no subfolders. See the older built-in frcData for reference: https://github.com/Mechanical-Advantage/AdvantageScope/tree/v2.3.0/frcData.
Once the 2024 betas are ready, all of the assets in the 2023 format will be upgraded automatically so don't worry about developing with the 2023 format right now.

Thank you! This was able to resolve my issue. The problem was that the online documentation was for a more recent version of AdvantageScope than the most recent release. This could be resolved temporarily by adding a disclaimer at the top of the "Custom Assets" page. I believe I was also testing using assets created for the current version on main rather than the version of AdvantageScope that I was using.
Thank you so much for your help.

Disclaimers added! 👍
Thanks for the suggestion.


Create the kusto query on log analytics.

To create the invocation bar chart from the log analytics data

customMetrics |
where name == 'ServiceBusTrigger Count' |
project valueSum, timestamp |
render columnchart


Python3 : Numpy scalar types "not iterable"; specific instance ndimage.filters.gaussian_filter

With some toy code to produce Perlin noise, I came across a discrepancy between Python 2 and Python 3 behaviour. This applies for numpy 1.10.1 and scipy 0.16.0, under Python 2.7.10 and Python 3.4.3.
X = np.random.normal(size=(256, 256))
perlin = perlin = np.zeros_like(X)

for i in 2**np.arange(6) :
    perlin += nd.filters.gaussian_filter(X, i, mode="wrap") * i**2

This works great under Py2. For Py3, I have to cast i as int(i) in the call to gaussian_filter, or I get TypeError: 'numpy.int64' object is not iterable.
I'm unsure if this has been seen before; sorry if so. Any thoughts appreciated.

It looks like the issue occurs in ndimage._ni_support._normalize_sequence, which contains the following snippet:
if isinstance(input, integer_types + (float,)):
    normalized = [input] * rank
else:
    normalized = list(input)

In python 3 the input is being sent to the else clause, which raises the error (the value of integer_types is python-version dependent). Will try to learn more.
Also, possibly related: https://github.com/numpy/numpy/issues/2951

I'll guess that 2 is converted to int64 because it is a long int in Python 3. One solution might be to use numbers.Integral instead of integer_types, that should get all the numpy integer types.

That's a nice solution (changing it to
if isinstance(input, (Integral, float)):

fixes the issue), but it wouldn't work for numpy < 1.9 or python < 2.6, right? Would it be okay to break the back-compatibility? (Scipy's _init_ file currently warns that numpy >= 1.6.2 is recommended, not sure about recommended python version.)
Perhaps the definition of integer_types could be further refined to use numbers.Integral when it's available and do something more complicated when it's not (with the aim of being easily able to remove the complicated parts when old versions get deprecated)?

Python < 2.6 we already don't support, so that's not an issue. And 2.6 itself will go at some point in the near future I expect - Matplotlib already dropped support for it I just read.
Things need to work for numpy's before 1.9 though. We can drop 1.6.2 support soon when needed, but dropping also 1.7 and 1.8 support is a bit too much.

However, if something that's broken now can be fixed easily by a >=1.9 feature, then it may be OK to do if npversion >= 1.9 then use_working_code else dont_change_a_thing. I didn't look into this issue, so not sure if a fix for older numpy's is that hard, but a partial fix is better than no fix.

This issue also affects interpolate.BPoly.from_derivatives (AFAICT there are only three functions which use integer_types, the remaining one is io.netcdf_file._write_values, which appears to be fine.) The issue with BPoly.from_derivatives can be seen by running
x = np.linspace(0, 1)
order = np.int64(1)
p = BPoly.from_derivatives([0, 1], [[1, 2], [3, 4]], orders=order)

which will raise the same TypeError. Running it with orders = 1 works fine. To me the best fix for _normalize_sequence seems to be
if hasattr(input, '__iter__'):
    normalized = list(input)
else:
    normalized = [input] * rank

with a similar fix for BPoly.from_derivatives. Will write a patch for people to look at.

(While the numbers.Integral solution feels more elegant to me, testing for an __iter__ method should work for all versions of numpy.)


Change default podManagementPolicy to Parallel for Zookeeper

Motivation

For speeding up startup, podManagementPolicy should be set to Parallel for Zookeeper

Modifications

set zookeeper.podManagementPolicy to Parallel in the default values.yaml file.

Note for future googlers, this change will break a helm chart upgrade if you installed the chart before this change due to podManagementPolicy in a StatefulSet being an immutable field.

Error: UPGRADE FAILED: cannot patch "pulsar-zookeeper" with kind StatefulSet: StatefulSet.apps "pulsar-zookeeper" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

Set it back to the previous version with the below in your custom values file which I don't think does any harm for existing installs
zookeeper:
  # Changed to Parallel by default in https://github.com/apache/pulsar-helm-chart/pull/203
  # However since we installed the chart prior to this and this field is immutable
  # set it back to the old value.
  podManagementPolicy: OrderedReady

@rmyates - thank you for posting this message. This change was intended to be backwards compatible, so it is valuable feedback to know it wasn't. Now that we've changed the default, I don't believe we can go back without also breaking new deployments that inherited the Parallel default.

@michaeljmarshall No worries. I think keeping the change for new users/installs is fine and does make sense looking at the k8s docs.
It's just one of those things where an immutable field bites you every now and then! I suspect they aired on the side of caution with changes to existing StatefulSets given the potential for destruction.


parallelize 2i input to pipe mr (AZ798)

As with the earlier change to how bucket-key-lists are injected into a
MapReduce pipe, this patch makes the same change for secondary index
inputs: a short pipe is started, and the riak_pipe_qcover_fsm is used to
generate inputs for the MR pipe.
The appenv variable 'mapred_2i_pipe' is provided, and defaulted to 'false'
to provide rolling-upgrade compatibility with 1.0 clusters (which do not
have the riak_kv_pipe_index module).
This patch does introduce a bit of duplication with riak_kv_pipe_listkeys.  This duplication keeps the changeset and upgrade clean.  A merge could be possible if it annoys too much.

+1 to merge


Convert ids to classNames, use refs

@mfix22 I think the api needs to be redeployed to now. Logs say the last deploy was August 23rd, and this change was August 27th I think. Puppeteer is still looking for #export-container and therefore returning the error Error: Evaluation failed: TypeError: Cannot read property 'offsetWidth' of null

Thanks @payers1 I will take a look immediately


Including .mustache file with JSON data in higher level pattern.

I would like to have a generic  THEAD that gets its THs from JSON included in a Table Organism that is included in a Page. Same with a couple TBODY rows.
<thead>
      <tr>
             {{#tableheadercells}}
            <th {{htmlAttr}} class="{{cssClass}}">{{{content}}}</th>
              {{/tableheadercells}}
          </tr>
</thead>'

Works fine on its own...
But I would like to use this above mustache partial in a page containing the entire table. Is this possible?
I've googled for an answer, but haven't found a good one. Seems like you could sequence html rendering order by pattern type and there would be no JS confusion and PHP could even be easier since you can have more control over processing order in relation to pattern hierarchy... or am I just too new to this whole thing?

Crickets

@madpixl -
You don't get an answer in 17 hours and decide you need to follow-up with "Crickets?" Nice... A page cannot see the JSON for included organisms and molecules.

Sorry if crickets comment was inappropriate. I don't go on the public Github as often and our internal network is more active...
I'm trying to promote use of Patternlab as our reference html/css pattern library and feeling a lot of pressure to make it work for us. If it works I'm the hero, if not I've wasted a lot of time.
I already found out that the JSON data was not seen, but I was wondering if there was a way around that. Maybe forcing a render prior to inclusion for higher level objects, so the basic deconstruction of patterns could be more consistent.
I guess I will have to dig a little deeper to figure out if I can make it happen on my own. My premise was to share the JSON file developers use with DRULEs to render the Angular UI for our HTML-only reference application so that the partials could be leveraged more easily, and we could share more code on a variety of levels.

@madpixl Here's an example that might be of use to you. You can set up initial data in data.json and override that data at the page level. You can wrap individual instances of a pattern in a name, then target the data in the JSON.
Please keep in mind that this is an open-source project that is driven by people who have full-time jobs, families, and other responsibilities.

@madpixl -
A pre-render would defeat the purpose of Pattern Lab. A pattern could have one and only one block of data used to render it. No overrides possible.
The thing you could do is muck with how PL builds the global data store from data.json. Rather than just read in that one file you could scan all of the patterns for their individual JSON files and tack their data onto that global store of data. You'd have to be careful with namespacing but it's doable.
Really not sure we'll be much more help. Sorry.

Thx.


Some scenarios did not survive previous migrations

While migrating all protected scenarios from production for the heat changes (using etm-tools query-scenario) there were about a 100 scenarios ETE wasn't able to query or load.
After some digging by @michieldenhaan we found these scenarios still carried old (2018) names for sliders, contained sliders that were no longer present, or contained strange values. It seems these scenarios did not survive previous migrations.
We are unsure about the cause, and will probably leave it like it is. Very very strange...
These are all the id's of the scenarios that won't load: still_broken.txt Maybe there are scenarios in there that are valueable to any of you?
notifying @antw @ChaelKruip @redekok @jorisberkhout @marliekeverweij @DorinevanderVlies @MartLubben @AlexanderWirtz

Can it be that we protected these scenario's after they were originally made?
So that it is possible that there was a migration between the moment that the scenario was made and that the scenario was protected?

F


Reworked ignore rules and range handling

Resolves #952

These changes raise another issue https://github.com/aquasecurity/tfsec/issues/963


Update performance test results for Ballerina v1.0.5

Purpose
This PR updates the performance test results for Ballerina v1.0.5.
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #20471 into ballerina-1.0.x will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff                @@
##           ballerina-1.0.x   #20471   +/-   ##
================================================
  Coverage            20.51%   20.51%
================================================
  Files                   52       52
  Lines                 1394     1394
  Branches               216      216
================================================
  Hits                   286      286
  Misses                1095     1095
  Partials                13       13

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7440a10...6f8baab. Read the comment docs.


[PDI-17935] Dimension lookup / update step - Data type error:

@pentaho/tatooine @pentaho-lmartins
@mbatchelor @lucboudreau
Using MySQL, the ResultSet was returning an invalid COLUMN_SIZE ( Pentaho length ) of 22.

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -Dsurefire.runOrder=alphabetical -Daudit -amd clean install

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  core/src/main/java/org/pentaho/di/core/row/value/ValueMetaBase.java
  core/src/test/java/org/pentaho/di/core/row/value/ValueMetaBaseTest.java

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes (click to expand)
org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change: -0.4% :small_red_triangle_down:
Complexity Change: -0.5% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -0.8% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Method Change: -1% :small_red_triangle_down:

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Branch Change: -2.6% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.gpload.GPLoadMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.8% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change: -0.1% :small_red_triangle_down:


Add a test to verify the test code in question and solution are identical

In case people accidentally changed on but forget about the other

How about just put the test code in a standalone `check.py' instead?

I've decided to keep code in one file to simplify the working of creating new challenges (as it's easier to test).


Update from <IP_ADDRESS> to <IP_ADDRESS> stuck

We have solved described issue below, and this is just a record of that described issue and could serve as a guide for anyone experiencing the same.
Describe the issue
We recently update our Dash Core from <IP_ADDRESS> to <IP_ADDRESS> for the reason that our dashd stopped syncing (we believe this might be the original issue). After updating and checking the logs we found out that dashd is stuck on block 1088639 or block hash<PHONE_NUMBER>0000112e41e4b3afda8b233b8cc07c532d2eac5de097b68358c43e.
We tried to restart dashd with --reindex-chainstate and --reindex with both instances also getting stuck on the same block.
I remember we had a very similar issue before when BCH forked with BSV. So we tried the same solution by running the command:
dash-cli reconsiderblock<PHONE_NUMBER>0000112e41e4b3afda8b233b8cc07c532d2eac5de097b68358c43e

And this seemed to worked and our dashd started to sync immediately without even restarting it.
Expected behaviour
Dashd should continue to sync.
Actual behaviour
Dashd got stuck upon syncing.
What version of Dash Core are you using?

Upgrade from <IP_ADDRESS> to <IP_ADDRESS>

Machine specs:

OS: Ubuntu 18.04.1 LTS
CPU: Intel(R) Xeon(R) CPU E5-1660 v3 @ 3.00GHz
RAM: 125G
Disk size: 4TB
Disk Type (HD/SDD): HD

Any extra information that might be useful in the debugging process.
A portion of the debug.log can be seen at https://pastebin.com/dzNr5NUR

Encountered a similar issue when upgrading from v0.13.3 to v<IP_ADDRESS>, however my node showed no indication of being stuck at a specific block.
Had to do a writeup, might help someone (https://gist.github.com/Auronmatrix/8d3e4caffdb650c9d7ab485772656ecc)

Not sure what exactly happened, but seemed like my node was part of a separate network segment that continued to function but was not actually showing the correct data.

@Auronmatrix yes, pre-0.14 nodes forked off because they were not able to recognize coinbase extra payload v2 (https://github.com/dashpay/dips/blob/master/dip-0004.md#coinbase-special-transaction) which was activated via bit 4. Few miners and explorers did not upgrade and kept extending/displaying the wrong chain which caused a confusion.
Nice post mortem btw 👍

Was the fork to be expected? Would be good if there was some warnings (maybe as part of the 0.14 release notes) for those running pre-0.14?
From the logs nothing was obvious and things just seemed business as usual except for the data I got from the RPC making no sense.

@Auronmatrix Yes, the fork was expected. There is a small note that downgrade to 0.13 is only possible until dip8 activation https://github.com/dashpay/dash/blob/master/doc/release-notes/dash/release-notes-0.14.0.md#downgrade-to-versions-01300---01330 and we communicated that if you do not upgrade you are going to be forked off in all media channels as well I think. But I agree, we should probably make it more obvious in release notes if new upgrade is a hard fork or not. Smth to consider for future release notes I guess @codablock @nmarley

I would not have understood downgrade to mean staying on the same version while the DIP activates.
Only took that warning to mean upgrading to v0.14, would cause changes to the chain state/format that would not be possible to undo without a full reindex if you wish to go back to a pre-0.14 version.
Looks like I might not have been the only one :wink:. Would be great to be super explicit in the release notes, that's primarily what we follow very closely.

In order for this to work, I also had to run:
dash-cli invalidateblock<PHONE_NUMBER>0000112e41e4b3afda8b233b8cc07c532d2eac5de097b68358c43e

And restart the daemon. Then it appeared to rewind to the fork point (progress going backwards) and resyncing from there.

Issue solved


Racer: Crashed (failed to read env var?)

macOS Sierra (10.12)
VSCode: 9e4e44c19e393803e2b05fe2323cf4ed7e36880e 1.6.1
RustyCode: 0.19.1
Last command:

Racer Output:
RUST_SRC_PATH environment variable must be set to point to the src directory of a rust checkout. E.g. "/home/foouser/src/rust/src"
Racer Error:

racer works fine when I invoke it so I do not know what is broken here:
tmba:sql_lexer thill $ echo $RUST_
$RUST_BACKTRACE         $RUST_NEW_ERROR_FORMAT  $RUST_SRC_PATH

tmba:sql_lexer thill $ echo $RUST_SRC_PATH
/Users/thill/.multirust/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src

mba:sql_lexer thill $ racer complete std::io::B
MATCH BufReader,50,11,/Users/thill/.multirust/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/io/buffered.rs,Struct,pub struct BufReader<R>
MATCH BufWriter,306,11,/Users/thill/.multirust/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/io/buffered.rs,Struct,pub struct BufWriter<W: Write>
MATCH BufRead,1203,10,/Users/thill/.multirust/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/io/mod.rs,Trait,pub trait BufRead: Read
MATCH Bytes,1600,11,/Users/thill/.multirust/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/io/mod.rs,Struct,pub struct Bytes<R>

These commands were ran in the terminal integrated into vscode, fwiw.

You see #201 ? Maybe you have same problem?

This problem is similar in that using the config values (with absolute path) 'fixed' the issue, but RustyCode is not reading the environment variables as it suggests it does (something in the attempt to do so is failing) because my variables are set and work fine if used outside of RustyCode.

Yesterday I tried whether environment variables (RUST_BACKTRACE=1 in my case) would be forwarded to a cargo task --- they are not. It's quite likely vscode does not forward your env vars automatically to racer either. Seems like one would have to do this manually.

I needed to install src for specific rust version, e.g.
rustup component add rust-src --toolchain nightly-2017-04-16-x86_64-pc-windows-msvc
and then I got ~/.rustup/toolchains/nightly-2017-04-16-x86_64-pc-windows-msvc/lib/rustlib/src sources.


WatchOS 2

I'm wondering if setting a keychain key on iOS and trying to access it on Apple Watch is possible using this library. With WatchOS 2, Apple doesn't allow keychain sharing (using the capabilities but as far as I understand, the Apple Watch app has access to the keychain natively.
Can you elaborate on this?

Very good question, I do not know the answer, sorry.

@evgenyneu but you just updated your library to support watchOS or? Yet the demo project doesn't actually contain any watch extension. can you elaborate on what you support?

Oh, yes, you can use this library to write text to the Keychain and read form it on watchOS 2. I did a quick test with keychain.set("hello world", forKey: "my key") and keychain.get("my key") on Apple Watch and it worked. I do not know how and if you can share it with your container app though.

Can you set a key on iOS app and see if you can get it on Apple Watch?
Without keychain sharing or anything.

On Oct 19, 2015, at 6:11 PM, Evgenii Neumerzhitckii<EMAIL_ADDRESS>wrote:
Oh, yes, you can use this library to write text to the Keychain and read form it on watchOS 2. I did a quick test with keychain.set("hello world", forKey: "my key") and keychain.get("my key") on Apple Watch and it worked. I do not know how and if you can share it with your container app though.
—
Reply to this email directly or view it on GitHub https://github.com/exchangegroup/keychain-swift/issues/8#issuecomment-149362533.

you may need to enable iCloud Keychain in settings

On Oct 19, 2015, at 6:12 PM, Ace Green<EMAIL_ADDRESS>wrote:
Can you set a key on iOS app and see if you can get it on Apple Watch?
Without keychain sharing or anything.

On Oct 19, 2015, at 6:11 PM, Evgenii Neumerzhitckii<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Oh, yes, you can use this library to write text to the Keychain and read form it on watchOS 2. I did a quick test with keychain.set("hello world", forKey: "my key") and keychain.get("my key") on Apple Watch and it worked. I do not know how and if you can share it with your container app though.
—
Reply to this email directly or view it on GitHub https://github.com/exchangegroup/keychain-swift/issues/8#issuecomment-149362533.

Sorry I do not have time to test it right now. It may require modification to the library to support the keychain access groups (also asked in #6).

Or one can use WatchConnectivity and send the text form the watch to the container app which will save it the keychain.

WC only works on iOS9, so it limits a little. I would really appreciate it if you can try this scenario out. If it works, it makes your  library very appealing. Setting access to Always is not hard (at least hardcoded for this test purpose)

Yeah, I agree, it would be very useful to people. I will see if I can try it on the weekend. You are very welcome to try it on our own and post your results here.

I tried it but I don't see iCloud keychain support on simulator. I also don't have an Apple Watch to test it.

Cool, if I have time I will test it on Apple Watch device, but it won't happen soon.

then unfortunately it will be useless for me by then. Anyways ill see if RSSecrets has better support

This one looks good: https://github.com/kishikawakatsumi/KeychainAccess

it actually has potential. Thanks

Yes it looks like it has the sharing features that you need. I have also added a list of Keychain libraries to the readme:
https://github.com/exchangegroup/keychain-swift#alternative-solutions
Thanks for your feedback.

It looks like right now there is no way for a watchOS 2 app to share keychain item with the paired phone right now. Confirmed it by testing on Apple watch device. Keychain sharing and access group do not allow sharing data between the watch and the phone. Could not make the watch work with iCloud keychain either.

Apple employee confirms that: https://forums.developer.apple.com/thread/5938

Thanks for the feedback @evgenyneu. It's unfortunate that they don't support either one. This only complicates development for watch app

Closing the issue, feel free to reopen if needed.


Getting 'ascii' codec error, possibly created by colorama

Traceback (most recent call last):   File "/usr/local/bin/evillimiter", line 9, in <module>     load_entry_point('evillimiter==1.3.0', 'console_scripts', 'evillimiter')()                                                                          File "/usr/local/lib/python3.5/dist-packages/evillimiter-1.3.0-py3.5.egg/evillimiter/evillimiter.py", line 157, in run                                  IO.print(get_main_banner(version))   File "/usr/local/lib/python3.5/dist-packages/evillimiter-1.3.0-py3.5.egg/evillimiter/console/io.py", line 33, in print                                  print(text, end=end, flush=flush)   File "/usr/local/lib/python3.5/dist-packages/colorama/ansitowin32.py", line 41, in write                                                                self.__convertor.write(text)   File "/usr/local/lib/python3.5/dist-packages/colorama/ansitowin32.py", line 164, in write                                                               self.wrapped.write(text) UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-16: ordinal not in range(128)
How to solve this. Running project from docker container.

The problem seems to stem down from issues with python ascii encoding
Solved it by adding the next line to /etc/environment
PYTHONIOENCODING=utf8
If anyone faces the same issue in the future, simply do
echo PYTHONIOENCODING=utf8 >> /etc/environment


Update react-dev devDep. Change script color.

Closes #29
Changes
Steps to Test

Merge conflicts @shvagen


update-winget task is changing multiple files with PRs into winget-pkgs

There is a bug in the update-winget task that is making it open PRs against the winget-pkgs repo with multiple manifest files (there should only ever be one updated file per Pull Request).
This looks to be failing in this line of this if block.

Is this fixed by #107?

I believe it is fixed by #107

Yep! It was - sorry, for some reason I don't seem to be able to link issues to PRs in this repo.


Doodles: attempt to add ESLint, round 3

Kind of a conglomeration of #305 and #325. In progress:

[x] Use ESLint as the runner
[x] Strip out DT-specific codepaths per https://github.com/microsoft/dtslint/issues/300#issuecomment-903310045
[ ] Bring in any rules already converted to ESLint, such as from #305
[ ] Run the rest of the rules from @typescript-eslint/eslint-plugin-tslint
[ ] Make the output rules behavior as similar to today's ruleset as possible

A few notes:

Porting --expect-only would require running ESLint separately on every file or folder of files. Is that something we really want to keep?

In more detail: ESLint natively supports a unique config per-file and there's no way in the ESLint API to disable all rules except for an allowlist. We'd have to manually run per-file instead of using its native concurrency system, which will pending review in https://github.com/eslint/eslint/issues/3565 <- https://github.com/eslint/rfcs/pull/42 now.

ESLint's --rulesDir ecosystem works a bit differently from TSLint's --rules-dir and likely will make it harder to position a dtslint as the "only" dependency to be installed.

eslint-plugin-local-rules exists as a result.
More discussion is available in https://github.com/eslint/eslint/issues/8769 -> https://github.com/eslint/eslint/issues/9484

I need to think a bit more; posting this up for now as a scratchpad... will move all these todo-style notes into a <details> tag and make a real PR description once that's done. 🤔

i have an unpushed branch somewhere with a bunch of the rules converted, ill push that tonight in case its useful 👍

Closing as this has aged away. If you the reader would like this code, feel free to take it and send your own PR. ❤


PC-1350 port with the AS61860 assembler d8ced06

Added pc-1350 subdir with updated asm sources, updated xm2osk.pl, updated readme, new ihx converters, and new retargeting file target.h intended for porting to other SHARP PC.

First of all, sorry for the late reply, and thanks a lot for your work!
I'm now thinking that since your version also builds for the PC-1403, we
could just replace the original version, instead of adding it in a new
subdir. Replacing yasm with as61860 is fine by me. The only issue I see
is that this will deprive Windows users of a convenient workflow,
because of the dependency on make. Then again probably nobody is using
this anyway, Windows or not. Either way, if you'd be willing to change
the PR so the original and your version are merged, that'd be great.
Otehrwise, I'll take the PR in its current form and clean up later.
In any case, I'd ask you for a couple of changes before pulling, though.
I think play.bas and play.bin should be removed from version control.
Probably easiest add a .gitignore for all the build artifacts at
toplevel. (I had only just started to learn programming back when I
wrote this, and I didn't know better back then).
Also, how about generating target.h via the makefile, so that users
can set the target when invoking make, ie. make and make pc1403 will
build for 1403, and make 1350 will build for the latter machine?

I'm now thinking that since your version also builds for the PC-1403, we
could just replace the original version

I will change the PR.

I think play.bas and play.bin should be removed from version control.

If we retarget more machines in one dir, then sure it makes sense to remove these.

Also, how about generating target.h via the makefile, so that users
can set the target when invoking make, ie. make and make pc1403 will
build for 1403, and make 1350 will build for the latter machine?

That's a nice idea. I will make the change. I will work on that next week.

I've updated the makefile and readme to support specific machine builds. I've also reviewed the code again and made a few minor changes (see below). Running the sound over a speaker connected to a PC-1350 works, but the sound is not 100% the same as your post here https://soundcloud.com/irrlicht-project/osaka-ii-test-sharp-pc-1403
Because I worried that it wasn't exactly the same, I double checked the code and compared it to your original but could not find any differences that could be a problem. I even put back anim 0 which turns off the display. Strangely, that affects the sound too, so I'm now more picky to keep things the same. I've introduced macros BZHI and BZLO to set the orim and anim values of the buzzer port throughout main.asm, so experimentation is a bit easier. One explanation of the difference on my side is that my CE-126P cassette interface output port has been more quiet than it used to be, at least 50% less volume. Perhaps an opamp is not working properly after 45 years.

That looks excellent to me. Thanks a lot for your hard work!


How to train this model in a new corpus?

How to train this model in a new corpus? I use this model in en->ch, but I find this result is not good in fact, so I abtain a large bilingual corpus and want to train this model again.

Why not just re-run the training command, just as you did on the smaller dataset?

how did you resolve this problem??

并没有解决啊~
------------------ 原始邮件 ------------------
发件人:                                                                                "clab/fast_align"<EMAIL_ADDRESS>发送时间: 2020年9月9日(星期三) 中午1:32
收件人<EMAIL_ADDRESS>抄送: "zenRRan"<824203828<EMAIL_ADDRESS>主题: Re: [clab/fast_align] How to train this model in a new corpus? (#50)
how did you resolve this problem??
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

可以python force_align.py -h看下，里面给的命令保存的文件应该就是模型，在新语料上训练就可以，不知道我说的跟你问的意思一不一样

确实 "result is not good in fact"。
请问问题解决了吗？或是有其他的代替方法吗？


Unable to resolve module crypto

Error: Unable to resolve module `crypto` from `node_modules/dropbox/dist/Dropbox-sdk.min.js`: crypto could not be found within the project.

I got this error in my react-native project that I install locally today. Also, I installed dropbox(5.2.1) today and I remember it was working well when it was 4.0.30
Please let me know if I should downgrade it or which mistake that I made
Thanks

The Dropbox API v2 JavaScript SDK doesn't officially support React Native, so I'm afraid I can't be of much help with this. I'll send this along as a request for support for React Native, but I can't promise if or when that might be implemented though.
(That said, this post looks similar and may be helpful.)

We don't have any plans to support React Native in the near future so I'm closing this out, but hopefully that post I linked to was helpful.


Code cleanup

Codecov Report
Base: 93.18% // Head: 93.18% // No change to project coverage :thumbsup:

Coverage data is based on head (702198e) compared to base (bbe8199).
Patch coverage: 100.00% of modified lines in pull request are covered.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@           Coverage Diff           @@
##           master     #287   +/-   ##
=======================================
  Coverage   93.18%   93.18%
=======================================
  Files          17       17
  Lines        1159     1159
  Branches      208      208
=======================================
  Hits         1080     1080
  Misses         52       52
  Partials       27       27

Impacted Files
Coverage Δ

...enerator/AttributeFilter.RequiredAttributeNames.cs
100.00% <100.00%> (ø)

...Generator/AttributeFilter.SkippedAttributeNames.cs
100.00% <100.00%> (ø)

src/PublicApiGenerator/AttributeFilter.cs
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Move VFS interface into a common platform library [phab:D332]

Created by schroederc at 2015-06-10 18:52:26:

fromberger commented inline (context lost during migration) at 2015-06-10 19:03:58:
Now that this is in its own package, please consider naming this vfs.Interface (a la sort.Interface) since VFS is now redundant.

fromberger commented inline (context lost during migration) at 2015-06-10 19:03:58:
This comment needs updated. It's no longer in context.
In particular, the indexpack has particular requirements for what has to be implemented—those are no longer clear when reading this package in isolation, so I'd like to see it spelled out which operations must be implemented.

fromberger commented inline (context lost during migration) at 2015-06-10 19:03:58:
Better still, just make Open return a vfs.Interface, then you don't need this to force the compiler to check.

fromberger wrote at 2015-06-10 19:03:58:
Can you please give some context for why this separation is useful?
This wasn't intended to be a "generic" filesystem interface—which isn't really practical anyway. I was considering changing the storage interface used by the indexpack package anyway, so if you have a use for it elsewhere, I don't object to it moving—but I'm not super comfy introducing undocumented constraints with other packages without a good reason.

schroederc wrote at 2015-06-10 19:34:51:
Use vfs.Default in tools

schroederc wrote at 2015-06-10 20:48:09:
Rename vfs.VFS to vfs.Interface and create Reader/Writer interfaces

schroederc commented inline (context lost during migration) at 2015-06-10 20:51:14:
The indexpack package needs access to the underlying Archive

schroederc commented inline (context lost during migration) at 2015-06-10 20:51:14:
Done.

schroederc commented inline (context lost during migration) at 2015-06-10 20:51:14:
Done.

schroederc wrote at 2015-06-10 20:51:24:
Fix comments

fromberger commented inline (context lost during migration) at 2015-06-10 22:12:14:
:)

fromberger accepted this revision at 2015-06-10 22:12:14

schroederc closed via commit 02cfb740d6611de1af8163cedb78a06b026b472a at 2015-06-10 22:27:33


[WIP] Improve some test case

Improve some test case for BlockchainService. and closes #1269

you need to set the lib to do the test
发自我的 iPhone

在 2019年3月19日，12:35，todo[bot]<EMAIL_ADDRESS>写道：
This logic maybe not exact.
https://github.com/AElfProject/AElf/blob/473934f2f821edfd42601d78fa78aedfce231b68/AElf.Kernel.Core/Blockchain/Application/IBlockchainService.cs#L195-L200
This comment was generated by todo based on a TODO comment in 473934f in #1273. cc @AElfProject.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


feat: show globe view on map load

Description
Show a view of the whole earth on map load.
Issue
fixes #347
Checklist:

[x] My commit message follows the Conventional Commits specification
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have tested my code
[x] My changes generate no new warnings
[x] My PR is rebased off the most recent develop branch
[x] My PR is opened against the develop branch

Additional comments

Set interactiveLayerIds after the map loads and the grid is calculated.

Alert Reviewers
@codynhat @gravenp

Do we want the globe to also be on the testnet?

Maybe it's better to leave it zoomed in on testnet for easier testing. What do you think?

Yeah, if we can keep testnet claims focused in NYC by using an env variable that would be perfect.


Authentication of content not of users

To ensure a positive and respectful learning environment for all, DEV-G have to  implements a content review process. This process involves a team dedicated to reviewing content posted by both teachers and students to identify any potential issues that might not align with DEV-G's values. the content posted by teachers and students is subject of respect of DEV-G , so a system that manually check the content of teachers and inform the admin if find somthing that raise questions on DEV-G reputation

Assign this issue to me i want to work on this under ?GSSOC'24

Hi @surendar-05 can you explain the approach and flow that you will use to do this task? Just a brief approach.

Hi @surendar-05 Waiting for your response. Are you excited to work?

I am currently working on the issue as well as i am making detailed work flow for my issue

@surendar-05 Great!! Best wishes..


Internal publish

When embedding this server, can the application doing the embedding publish a message to a topic directly?  If not, can that be added?

Hi @wz2b - currently this isn't supported, but it is on the roadmap (I got a bit busy with other life things). I can probably get this implemented in the next few days if you'd like?

Sure.  Would you also need the ability to 'create topic' or would publishing do that automatically?   My alternative is I can make a loopback connection over localhost but that's messier if there is a way to expose this capability in Server.
If you can expose a server.Publish(topic, content) or something similar, then I will help test.

I was thinking about this earlier this morning as there are a few ways to do it. In either case, there is no need to 'create topic' as the topic creation is handled when a) a client subscribes to a given topic, or b) a client publishes a retained message to a given topic. So just providing the ability to publish should be sufficient.
I am thinking of something like: Publish(topic string, message []byte, retain bool) (messageID int, err error), which would directly be issued out to all topic subscribers. I have omitted the QoS byte as the embedder is directly interacting with the server, making QoS redundant in this case.
If we are adding inline publishing I think it also makes sense to provide an ability to read messages which have been received. The most useful solution would be to expose a channel of server.Messages to the embedding code which will stream the incoming messages while the channel is connected. The embedding service can then operate on this from a goroutine or worker pool. I will look at what I can do to expose a topic filtering function for the embedded service to use also.
Does this sound like it could work?

It wouldn't return an error as it's technically correct behaviour (you can publish a retain=1 message to a topic with no subscribers, who will then get the message when they subscribe to the topic). In theory it's possible to return a boolean to indicate if there were or weren't any subscribers at the time of publishing, but it'd take a bit of refactoring so it depends on your use case and how important it is to you...
Otherwise yes, it would work exactly as normal :)

@wz2b Release v1.0.1 is up, it contains the the direct Publish method, and an onMessage event hook. Performance effects appear to be negligible.
I've added a usage example in https://github.com/mochi-co/mqtt/blob/master/examples/events/main.go
The direct publish method looks like this:
(https://github.com/mochi-co/mqtt/blob/master/server/server.go#L358)
// func (s *Server) Publish(topic string, payload []byte, retain bool) error
err := s.Publish("a/b/c", []byte("hello"), false)
if err != nil {
    log.Fatal(err)
}

and the event hook looks like this:
(https://github.com/mochi-co/mqtt/blob/master/server/events/events.go)
import "github.com/mochi-co/mqtt/server/events"

server.Events.OnMessage = func(cl events.Client, pk events.Packet) (pkx events.Packet, err error) {
    if string(pk.Payload) == "hello" {
        pkx = pk
        pkx.Payload = []byte("hello world")
        return pkx, nil
    }
    return pk, nil
}

I hope this helps. Let me know how it goes or if you have any questions or thoughts!

You're ahead of me!  Give me a couple of days and let me try it all out.  Once I'm successful I'll close the issue out.  Thanks for taking the time to do this!!!!  Hopefully it's useful to other people besides me, but even if it's not, I still appreciate it very much.

@wz2b Did this all work out?