Add partial pileup placement logic

Fixes #11621
Status
In development
Description
Introduce partial pileup placement logic described in #11621 It involves the following:

user place a request in form of {containerFraction: <int>, 'pileupName': <string>}
request is placed via PUT HTTP call to MSPileup data-service
MSPileup data service calls updatePileup API which by itself pass the call to MSPileupData manage and calls its updatePileupAPI
the updatePileup API performs necessary checks, creates cusomDID and creates new transition record with the following structure:

{containerFraction: int,
 customDID: string,
 updateTime: GMTimeInSeconds,
 DN: string}

In addition, we provide partialPileupTask in MSPileupTask class which is used during each polling cycle. This method performs the following logic:

it fetches pileup documents from MongoDB
it identifies the slope of container fraction
get rucio DIDs for our pileup document
get portion of DIDs based on ceil(containerFraction * num_rucio_datasets) formula
create new container DID in Rucio
call attachDIDs Rucio wrapper API with our set of DIDs and rses from pileup document
create new rules for custom DID
set expiration date (to be 24h) for already existing ruleIds from pileup document
add new ruleIds to pileup document

Is it backward compatible (if not, which system it affects?)
YES
Related PRs
<If it's a follow up work; or porting a fix from a different branch, please mention them here.>
External dependencies / deployment changes
<Does it require deployment changes? Does it rely on third-party libraries?>

Alan, it would be nice if you'll review it before you leave. I implemented initial logic but need to do some testing, etc. Since changes are touching different parts of WMCore, e.g. Rucio, MSAuth, etc. it would be nice to have initial review of these changes.

@amaltaro , I need additional input from you how we should handle PUT requests? We have two options here:

User will send {"containerFraction":<float>, "pileupName": <name of pileup document>} JSON with HTTP put
User will first whole JSON document from MSPileup, then change its containerFraction and send HTTP PUT request

In first case, we should perform additional look-up of corresponding pileup document, and therefore, few changes should be made to the code:

adjust MSPileup service to handle different PUT request schemas, so far we assume that user send valid pileup document, while in case 1 it would be a update small doc
add look-up of MSPileup document itself and only then perform updatePileup API call

Please clarify which scenario to adopt.

@amaltaro , I need additional input from you how we should handle PUT requests? We have two options here:

@vkuznet Valentin, apologies for missing this question.
My opinion on this is that we should not change the current behavior. As far as I can tell, we are using the updatePileup function:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/MicroService/MSPileup/MSPileup.py#L97C9-L97C21
which ends up calling MSPileupData.updatePileup function:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/MicroService/MSPileup/MSPileupData.py#L132
which itself already has a document lookup.
Said that, your option 1 seems to me to be the way to go, which I copy below as well.
"""

User will send {"containerFraction":, "pileupName": } JSON with HTTP put
"""

@amaltaro , I addressed your feedback. Few comments though:

I refactor MSAuth to better handle your concern about userDN. Now there is no such method but rather userInfo which is used without this codebase and we can use it now in MSPileup. I still think it is proper place to keep it
I added logic to handle incoming fraction request
I addressed your comment about usage of pileupName vs customName based on slope of container fraction
I addressed pylint concerns and squashed changes
the unit tests are also improved based on your request

I think this PR is ready for final review. Please note that @klannon is interested to merge this PR within this calendar year as I'm heading to holiday break starting this Thus and will not be available till Jan 4th. Therefore, please speed-up the review and if new comments will pop-up I can try to resolve them by Wed.
Said that, I checked jenkins report and failed tests seems not related to this PR (I also noticed something weird in jenkins tests, like you are not authorized messages which lead me to conclude that it may be some expired certificate are around which has avalanche effect).

@amaltaro , I addressed all your feedback suggestions. For majority of them I marked them as resolved, but I left few responses for those comments were I either disagree with your comment or it requires further clarification. Please see them on github as unresolved. Apart from that I addressed to my best effort all pylint/pep8 comments and checked unit test (those which still fail require some authorization and I doubt it is related to this PR). Please proceed with review.

@vkuznet thank you for the prompt action. Before I give it another review, can you please update the PR description with any information that might be helpful to understand what is provided in this PR?
I am particularly interested in seeing what the workflow is supposed to be in terms of synchronous vs asynchronous activity. Example, what happens when the end user makes an HTTP call to change a pileup document? Are we suppose to create the transition record? are we suppose to create the custom name? When running the async method for pileup resize, what is the assumption? what is supposed to happen in terms of data structure in and out?

Alan, I update description.

Alan, I addressed your new comments (but for some of the posting github web UI does not provide reply field). I think the page is quite long now. Said that, one of your comment (https://github.com/dmwm/WMCore/pull/11807#discussion_r1431713374) requires further clarification. The fraction and lastFraction has no definition if it is related to Disk or not. Both of them are containterFraction which we defined as "containerFraction": real number with the container fraction to be distributed (optional). Is it Disk or Tape fraction is not clear to me and requires further clarification. If you need such distinction I suggest to address it in different issue/PR as this one is too complicated already.

@vkuznet Valentin, I pasted your comment to the inline discussion (about fraction vs lastFraction) and also replied over there. Please let me know if you cannot find it and I am happy to paste it again in the main body of this PR.

[verbose mode enabled]
Here is the standard scenario that I have in mind:

end user creates a new pileup configuration with something like:
{'pileupName': 'blah1', 'customName': '', 'containerFraction': 1.0, 'transition': []}
the service will make the expected data  placement for pileupName
then the end user decides to change the container fraction with something like:
{'pileupName': 'blah1', 'containerFraction': 0.5}
the web-service will persist this information in the database as follows (according to the current state of this PR):
{'pileupName': 'blah1', 'customName': '', 'containerFraction': 0.5, 'transition': ['customDID': blah1-V1, 'DN': user1, 'containerFraction': 1.0, 'updateTime': time now]}
then MSPileupTasks will parse it and make a new data placement for the new fraction (now 0.5). Is it going to use the customDID created in the step above? Once it's all handled correctly, it's expected to have customName updated to customDID.

The final outcome of all of this corresponds to the expected behavior. However, I find the transition record to be misleading - as mentioned along the code - given that it has current and past information in the same record. For instance, customDID does not have a fraction of 1.0; fraction of 1.0 was not created during the updateTime neither by the DN defined in that transition.

Based on my previous comment, perhaps a better workflow would be:

end user creates a new pileup configuration with something like:
{'pileupName': 'blah1', 'customName': '', 'containerFraction': 1.0, 'transition': []}
the service makes the expected data  placement for pileupName
then the end user decides to change the container fraction with something like:
{'pileupName': 'blah1', 'containerFraction': 0.5}
the web-service persists information in the database as follows:
{'pileupName': 'blah1', 'customName': '', 'containerFraction': 1.0, 'transition': [{'customDID': blah1-V1, 'DN': user1, 'containerFraction': 0.5, 'updateTime': time now}]}
then MSPileupTasks parses it and make a new data placement for the new fraction (now 0.5). Once it's been successfully done, it updates the pileup document with something like:
{'pileupName': 'blah1', 'customName': blah1-V1, 'containerFraction': 0.5, }
If end user decides to change the container fraction once again with something like:
{'pileupName': 'blah1', 'containerFraction': 0.75}
the web-service persists information in the database as follows:
{'pileupName': 'blah1', 'customName': blah1-V1, 'containerFraction': 0.5, 'transition': [{'customDID': blah1-V1, 'DN': user1, 'containerFraction': 0.5, 'updateTime': previous ts}, {'customDID': blah1-V2, 'DN': user1, 'containerFraction': 0.75, 'updateTime': time now}]}
then MSPileupTasks parses it and make a new data placement for the new fraction (now 0.75). Once it's been successfully done, it updates the pileup document with something like:
{'pileupName': 'blah1', 'customName': blah1-V2, 'containerFraction': 0.75}

That means:

end user updates containerFraction, but initially it only reflects in the transition record. Once MSPileupTasks run, it will then get persisted in the top level pileup dictionary struct.
partial data placement uses the last transition record instead of top level containerFraction
transition records will be more meaningful, in the sense that each record has data associated to the very same action.

Please do share any alternatives/improvements/suggestions that you might have.

Alan, thanks for the input as it clarifies few things and broke others. Here is my observations:

with your proposal the update document diverge into two different schenarios:
a. if user provide update document with the same container fraction then document should be updated without transition record. For instance, the initial document had one set of campaigns, while a new one will add another campaign. Please note, that in both cases fraction can remain the same.
b. your example when container fraction changes during the update
Using your approach we should properly address update procedure. In case of (a) we update the document itself, in case of (b) we add transition record.

Said that, the flaw is a design choice of having separate update and updateTasks.
In my view to accommodate both (a) and (b) scenarios we should allow update to happen in both places in update and in updateTask. The former should first take current document values, e.g. containerFraction, and put them into transition. Then perform the update of the document itself, i.e. it will update its containerFraction. This way we preserve the current fraction value in transition. The updateTask will get document and place rules. But it will take last transition record and compare it with document container fraction. Once it is done it will update the document by adding new transition record. This update will happen only after it will succeed with rucio rules. Doing this way we keep update API as is and it will work identically in case (a) and (b). And, updateTask will be allowed (as it should be) to update document itself.
Please evaluate my proposal as I do not want to implement either yours or mine until we understand proper behavior. Please note, in my initial implementation I had mine implementation in place which you questioned and I removed upon your initial feedback. I still think it is a proper way of addressing this issue.

In my view to accommodate both (a) and (b) scenarios we should allow update to happen in both places in update (MSPileup) and in updateTask (partialPileupTask in MSPileupTask).

This is already how it works. Otherwise how would the pileup document get the rule ids created/deleted(?)

The former should first take current document values, e.g. containerFraction, and put them into transition. Then perform the update of the document itself, i.e. it will update its containerFraction. This way we preserve the current fraction value in transition.

This would be the best for update consistency. It has a drawback that I pointed out in my previous messages though (DN and fraction association is confusing). This problem: https://github.com/dmwm/WMCore/pull/11807#issuecomment-1863415866

The updateTask will get document and place rules. But it will take last transition record and compare it with document container fraction. Once it is done it will update the document by adding new transition record.

Here you lost me. Are you suggesting to have 2 transition records for the very same fraction update? One by MSPileup and one by MSPileupTasks

And, updateTask will be allowed (as it should be) to update document itself.

agreed! This is my first answer above (it already does).
I don't think we have converged with the best design yet, but I have another suggestion which I will make in a new post. Give me a few minutes.

Valentin, as we discussed this morning, here is a gist with the workflow proposal for how to handle partial pileup update:
https://gist.github.com/amaltaro/b4f9bafc0b58c10092a0735c635538b5
note that this keeps history of the data itself instead of MSPileup/MSPileupTasks.
If you have a gist for the microservice-based history, please do share it.

@amaltaro , this PR is ready for review. Please note the following:

there are lots of changes to many files, including Rucio mock, Authz, MSPileup codebase
I made Authz to work with local tests using localhost-user DN
I run all MSPileup local test locally, without this I doubt they will be finished at all. This is particular very challenging PR and relying on Jenkins was not an option (or it will takes years to run the logic of unit tests). I would like to stress this particularly on this use-case that we should be able to run unit tests locally.
I implemented full logic from your gist which we discussed extensively and provided unit tests listing every step of iteration
I found and filled all missing parts of Rucio mock APIs without them testing was impossible
Even though I implemented the whole logic I found it very complex (code wise). In my view we step away from Microservice nature and all our services are too big to be called Microservices. Said that, the challenging logic comes from separation of MSPilupe APIs with MSPileupTasks, i.e. asynchronous nature of processing documents. It would be much simple if we'll have everything under single service and made all operations atomic. I fear that further changes will be very difficult to implement to adopt to all possible use-cases.

@amaltaro , I addressed all issue you reported in your review except few items. Please reply to my observations/questions in place. In particular, In particular:

the custom Name pattern does not align with CMS dataset one
the MSPileupObj validate object rse list and therefore code provides them
create pileup should creates transition record per logic in gist and it is in disagreement with your comments
portion logic seems correct to me and you still insist that it is not valid, I followed the one reported earlier in this issue and I tried to outline it step by step. To avoid further back and forth comments I suggest that you write this snippet as you view it
I need further instructions what to do with Rucio logic which you pointed out as it is unclear from your review
I corrected createReplicationRule  API name (and its mock up counterpart)
Due to validation of RSE list in MSPileup object we still need to pass it to updatePileup API and there are two lists around expectedRSEs and currentRSEs. Please clarify which one to use

@amaltaro , I implemented new logic for increase/decrease fraction of pileup via separate functions. You can find appropriate commit in this PR. I also added https://github.com/dmwm/WMCore/issues/11863 issue to address separation of validation step from constructor of MSPileupObj. I also updated PR description with TODO to adjust MongoDB records. All unit tests are passed in my local setup and in Jenkins (though there is unrelated failed unit test from workqueue). Please have another look and hopefully we may converge and merge this PR.

Alan, thanks for review, I applied all changes you requested. I also edited and adjusted description where I included new issue for requested script to update records in MongoDB. I also added it and another one (separation of validation step, I made it optional though) to meta-issue. Feel free to make new review.

And I just noticed we merged this PR with 9 commits, instead of having them squashed before merge, sorry.


Feature request: Support updating non-master julia branches, like release-1.8

I don't believe this is currently possible @DilumAluthge ?

It's not currently possible, but it would be a good feature to add. I can take a look after JuliaCon.


Cursor should be a hand when hovering over the main interface buttons.

This makes it obvious that the buttons are clickable, by behaving consistently with links.

It should also be a hand when hovering over entries in the list of categories.

Actually, let's go with cursor: default for the categories. c6677be3022cfa3d0f6c55ebe59821febbe83680 and e2153dc2589eecc0d53df66ea7dfca63f8395407 resolve this issue.


Add asynchronous implementation of a LoRaWAN device

This adds an asynchronous implementation of a LoRaWAN device, where the
state machine is driven by async-await blocks, and allowing for an
asynchronous radio implementation.
The existing 'event driven' device remains untouched, but some types is
reused in the asynchronous implementation.
@ivajloip @lthiery I decided to try out creating a fully async version with async-friendly APIs instead of reusing the state machine. This also made the implementation code very simple to read also IMHO.
I've verified this works with several of the examples in drogue-device and will be moving them to use this

@ivajloip Would you mind merging this as well if it looks ok, as discussed in #41 ?

FWIW, I implemented @lthiery's suggestion of sharing create_join_request on the Credentials type as well.


OSError: Unable to open file (truncated file: eof = 96, sblock->base_addr = 0, stored_eof = 2048)

Hi,
thank you for your amazing work. I have already been using SuperGlue for previous work. Now I wanted to test your workflow for my own images and wrote a litte script based on the pipeline_SfM.ipynb.
The script run_pipeline.py throws an error at extract_features.main(feature_conf, images, outputs) and looks like this:
from pathlib import Path

from hloc import extract_features, match_features, reconstruction, visualization

images = Path('images/')

outputs = Path('outputs/sfm/')
sfm_pairs = outputs / 'pairs-exhaustive.txt'  # exhaustive matching
sfm_dir = outputs / 'sfm_superpoint+superglue'

feature_conf = extract_features.confs['superpoint_aachen']
matcher_conf = match_features.confs['superglue']

features = feature_conf['output']
feature_file = f"{features}.h5"
match_file = f"{features}_{matcher_conf['output']}_{sfm_pairs.stem}.h5"

extract_features.main(feature_conf, images, outputs)

As an error message I get:
[01/25/2021 17:13:40 INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
Loaded SuperPoint model
[01/25/2021 17:13:40 INFO] Found 26 images in root images.
  0%|                                                                                | 0/26 [00:00<?, ?it/s][01/25/2021 17:13:42 INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
Loaded SuperPoint model
[01/25/2021 17:13:42 INFO] Found 26 images in root images.
Traceback (most recent call last):
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\h5py\_hl\files.py", line 185, in make_fid
    fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (truncated file: eof = 96, sblock->base_addr = 0, stored_eof = 2048)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Prg\anaconda3\envs\hloc\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Prg\anaconda3\envs\hloc\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Prg\anaconda3\envs\hloc\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Prg\anaconda3\envs\hloc\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Prg\anaconda3\envs\hloc\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Prg\anaconda3\envs\hloc\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Prg\anaconda3\envs\hloc\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Arbeit\Workspace\Hierarchical-Localization-master\run_pipeline.py", line 21, in <module>
    extract_features.main(feature_conf, images, outputs)
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\autograd\grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "C:\Arbeit\Workspace\Hierarchical-Localization-master\hloc\extract_features.py", line 135, in main
    feature_file = h5py.File(str(feature_path), 'a')
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\h5py\_hl\files.py", line 408, in __init__
    swmr=swmr)
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\h5py\_hl\files.py", line 187, in make_fid
    fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
  File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py\h5f.pyx", line 108, in h5py.h5f.create
OSError: Unable to create file (unable to open file: name = 'outputs\sfm\feats-superpoint-n4096-r1024.h5', errno = 17, error message = 'File exists', flags = 15, o_flags = 502)
  0%|                                                                                | 0/26 [00:05<?, ?it/s]
Traceback (most recent call last):
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\utils\data\dataloader.py", line 872, in _try_get_data
    data = self._data_queue.get(timeout=timeout)
  File "C:\Prg\anaconda3\envs\hloc\lib\multiprocessing\queues.py", line 105, in get
    raise Empty
queue.Empty

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "run_pipeline.py", line 21, in <module>
    extract_features.main(feature_conf, images, outputs)
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\autograd\grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "C:\Arbeit\Workspace\Hierarchical-Localization-master\hloc\extract_features.py", line 137, in main
    for data in tqdm(loader):
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\tqdm\std.py", line 1167, in __iter__
    for obj in iterable:
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\utils\data\dataloader.py", line 435, in __next__
    data = self._next_data()
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\utils\data\dataloader.py", line 1068, in _next_data
    idx, data = self._get_data()
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in _get_data
    success, data = self._try_get_data()
  File "C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\utils\data\dataloader.py", line 885, in _try_get_data
    raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 17152) exited unexpectedly

Is it because I am not using CUDA but I already used SuperGlue without CUDA or am I missing something else? The  .h5 is always created with 1 kb but seems like it cannot be opened. I am running the code on Windows 10. Thanks for any help :)

Feature extraction should work well on CPU, but I have not tried on Windows. Did you try deleting the feature file and running the script again? Currently, it appears that the h5 file is corrupted if the extraction crashes.

Feature extraction should work well on CPU, but I have not tried on Windows. Did you try deleting the feature file and running the script again? Currently, it appears that the h5 file is corrupted if the extraction crashes.

Yes, I tried deleting only the file as well as deleting the whole folder structure. The error always reappears. I could try to reproduce this on a different machine but I would have to set up all the files at first.

Yes, I tried deleting only the file as well as deleting the whole folder structure. The error always reappears. I could try to reproduce this on a different machine but I would have to set up all the files at first.

Could you pinpoint where this crashes using a debugger? It's unclear at what line does h5py throw an error.

Could you pinpoint where this crashes using a debugger? It's unclear at what line does h5py throw an error.

I tried debugging using pdb. This is the output until the first error occurs:
-> extract_features.main(feature_conf, images, outputs)
(Pdb) n
[01/26/2021 10:22:07 INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
Loaded SuperPoint model
[01/26/2021 10:22:07 INFO] Found 26 images in root images.
  0%|                                                                                | 0/26 [00:00<?, ?it/s]> c:\arbeit\workspace\hierarchical-localization-master\run_pipeline.py(21)<module>()
-> extract_features.main(feature_conf, images, outputs)
(Pdb) n
[01/26/2021 10:22:18 INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
Loaded SuperPoint model
[01/26/2021 10:22:18 INFO] Found 26 images in root images.
OSError: Unable to create file (unable to open file: name = 'outputs\sfm\feats-superpoint-n4096-r1024.h5', errno = 17, error message = 'File exists', flags = 15, o_flags = 502)
> c:\arbeit\workspace\hierarchical-localization-master\run_pipeline.py(21)<module>()
-> extract_features.main(feature_conf, images, outputs)

At this point the file 'feats-superpoint-n4096-r1024.h5' was already created in outputs\sfm. Maybe a problem with setting the path in Windows 10? I already tried using absolute paths for the dataset. I also tried using the provided South-Building dataset and ran into the same error.

I tried debugging using pdb. This is the output until the first error occurs:
-> extract_features.main(feature_conf, images, outputs)
(Pdb) n
[01/26/2021 10:22:07 INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
Loaded SuperPoint model
[01/26/2021 10:22:07 INFO] Found 26 images in root images.
  0%|                                                                                | 0/26 [00:00<?, ?it/s]> c:\arbeit\workspace\hierarchical-localization-master\run_pipeline.py(21)<module>()
-> extract_features.main(feature_conf, images, outputs)
(Pdb) n
[01/26/2021 10:22:18 INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 4096, 'name': 'superpoint', 'nms_radius': 3},
 'output': 'feats-superpoint-n4096-r1024',
 'preprocessing': {'grayscale': True, 'resize_max': 1024}}
C:\Prg\anaconda3\envs\hloc\lib\site-packages\torch\cuda\__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  ..\c10\cuda\CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
Loaded SuperPoint model
[01/26/2021 10:22:18 INFO] Found 26 images in root images.
OSError: Unable to create file (unable to open file: name = 'outputs\sfm\feats-superpoint-n4096-r1024.h5', errno = 17, error message = 'File exists', flags = 15, o_flags = 502)
> c:\arbeit\workspace\hierarchical-localization-master\run_pipeline.py(21)<module>()
-> extract_features.main(feature_conf, images, outputs)

At this point the file 'feats-superpoint-n4096-r1024.h5' was already created in outputs\sfm. Maybe a problem with setting the path in Windows 10? I already tried using absolute paths for the dataset. I also tried using the provided South-Building dataset and ran into the same error.

This could be the same error: https://github.com/h5py/h5py/issues/1090

This could be the same error: https://github.com/h5py/h5py/issues/1090

I unfortunately don't have access to a Windows machine to try to reproduce the error. The pdb trace is too coarse, you would need to check what's happening in extract_features.

I unfortunately don't have access to a Windows machine to try to reproduce the error. The pdb trace is too coarse, you would need to check what's happening in extract_features.

Sorry for the late reply. At the moment I was not able to fix the issue but I am using a jupyter notebook now and it works for my purpose. Thanks anyway :)

Sorry for the late reply. At the moment I was not able to fix the issue but I am using a jupyter notebook now and it works for my purpose. Thanks anyway :)


Using two LMS1000 from one main thread

Hi everyone,
I have a simple question again. Is it possible to run/start two SICK LMS1000 from one main thread in my program? To do so, I have called the corresponding sick_scan_xd api methods that are needed for bring up the scanner twice. Here is what I do.
SickScanApiCreate(numberOfScannerParameters, scannerParameter) (called twice with coresponding params) SickScanApiInitByCli(apiHandle, numberOfScannerParameters, scannerParameter)  (called twice with coresponding params) SickScanApiRegisterCartesianPointCloudMsg(apiHandle, cb) (called twice with coresponding params)
If I run my main programm, I only see in the terminal log that one SICK TcpRecvThread is startet.
I also tried to use the SickScanApiInitByLaunchFile and gave the two nodes seperate names like I would do in ROS. But also in this case I only see one SICK TcpRecvThread comming up.
Am I doing something wrong, or is it not possible to bring up two Scanners from one main programm? If it is possible could you please decribe to me how?
I am using an Ubuntu 22.04 machine and a switch to connect the LSM1000. As you might have seen I use C++ and have no possibility to use ROS. The "main thread" mentioned above is just a simple C++ programm that receives the incomming scan data and process them.
Thanks in advance!

Thanks for your feedback. The sick_scan_xd-API currently does not support the (single- or multithreaded) use of 2 or more lidars in one process. We do not recommend doing so, since the sick_scan_xd library is not guaranteed to be threadsafe.
To run multiple lidars simultaneously, please use ROS or run sick_scan_xd in multiple and separate processes.


Allow video to continue playing when app enters background

Currently, when the app enters the background, the video stops playing. There should be a way to override this.

Sorry for the delay here. I'm doing some Spring cleaning, so due to inactivity I'll be closing this issue.
Please leave a comment on #88 or open a pull request if this issue persists. Thank you.


FLEX-2851 ~ Improves documentation regarding development image.

All is well.

All is well.

 All committers of the pull request should sign our Contributor License Agreement in order to get your pull request merged.1 out of 2 committers have signed the CLA.:white_check_mark: kevinsmeets:x: web-flow

All is well.


Post only project_id rather than entire project.

The link to delete a mail contained the complete project instead of just the project_id. But the project_id isn't required for deletion anyway.

I just spent all morning updating the plugin to be compatible with php 8,and a few other things, i'll check this out and see if its merge-able after i finish, should be.


Feature request: Focused specs (as Cedar does)

Hello,
It would be nice to have a support for Focused specs: I mark a spec I am currently working on as 'current' or 'wip' or 'fit' and it makes Kiwi to bypass all other specs when running a test suite.
Of course you know that Cedar already has it implemented and it is described in its README page,  Focused specs - it does exactly what I'm describing here.
Also, I've just reviewed this comment by @luisobo and now I see that together with my another issue https://github.com/allending/Kiwi/issues/289, he catched them both in a single comment.
Thanks!

Why is changing the code not a good way of implementing focused specs? In my mind, it's a far better user experience to "it -> fit" than to have to specify file name and line number. I've tried using @jerrymarino's plugin, which looks fantastic, but unfortunately currently crashes Xcode 6 when attempting to run a focused spec, or hangs without completion when attempting to run a focused file.

@kevinwo checkout the thread #236 which has a discussion about why we didn't add support for focusing specs by changing the source code.
There was an issue when using Kiwi on Yosemite that caused a hang. I'd recommend updating Kiwi and making sure you're build includes this fix.
The Xcode plugin just sets the environment variable KW_SPEC to indicate to Kiwi the focused spec before running the test suite. There isn't any known crashes caused by plugin alone.

Thanks @jerrymarino! The explanation in #236 makes sense. However, I still experience a crash on running a single focused spec with the plugin, and for a focused file I see a report of 0 tests being run. I just now went to open an issue on the plugin GitHub page, but do not see the option to do so.
I'm using Xcode 6.4 on Yosemite 10.10.4, and have tried using the plugin with both the podspec version of Kiwi (from 6 months ago it looks like from GitHub) and the current git codebase on master (3 months ago), both to no avail. I'm curious to see if you encounter the same issue with the example project I've posted up here:
https://github.com/kevinwo/KiwiSpecPluginIssueReproduce

There really should just be a built in way for Kiwi to do this. All plugin/terminal solutions that I've tried have been flaky at best. Sadly, running the whole test suite still ends up being faster.

Changing the scheme to have an environment variable is all fine and dandy, but I feel like I'm more likely to accidentally forget about that, and it's more time consuming.

I agree with @marklarr. One way or another, this should be a stable core feature of Kiwi.


bug(COMPONENT): TITLE Tool tip showing a black tooltip on top of another.

Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
13.2.3
Description
We are getting tooltips with the latest material version. the Diagonal blank tool-tip doesn't exist in 13.2.3.

Reproduction
Steps to reproduce:
1.
2.
Expected Behavior
only one tooltip.
Actual Behavior
The tooltip is showing two tooltips one is blank and is showing on top of the other.
Environment
Angular CLI: 13.2.5
Node: 16.13.1
Package Manager: npm 8.1.2
OS: win32 x64
browser edge and chrome
Angular:
...
Package                      Version
@angular-devkit/architect    0.1302.5 (cli-only)
@angular-devkit/core         13.2.5 (cli-only)
@angular-devkit/schematics   13.2.5 (cli-only)
@schematics/angular          13.2.5 (cli-only)

This isn't enough information to investigate. Can you provide a Stackblitz showing the issue?

Closing this issue due to no recent activity, if this is still an issue please create a new issue with a reproduction.


Show live status of publications

As a Publisher I can see status of publication immediately after I have pushed my package.

High level:

Get current status of the package from /source/{identifier}/status
Display message

loaded/registered/running: "your datapackage is being processed. Please, try again in 2minutes"
invalid: ?
failed: show error messages/logs
succeeded: render a page

Acceptance criteria

[x] I can see status of my just published package online

Tasks

[x] Refactor /:owner/:name route:

[x] if 404, check status

[x] display message based on status
[x] render the page if already succeeded

[x] if not, render the page

FIXED:

if package's status is failed we print out all logs (should be improved in the future but now we only have logs)
if package's status is SUCCEEDED we get the package from pkg-store and render the actual page
if package's status one of RUNNING, LOADED, REGISTERED;

@anuveyatsu I've  published something that errored  http://testing.datahub.io/zelima/nasa-tsv Seems it is not taking care of newlines making it impossible to read Trace

Please see new specstore API endpoints:

status endpoint (https://github.com/datahq/specstore#status) should be used while waiting for result (i.e. in a not succeeded nor failed).

logs in the response can be used to show the user the running logs

info endpoint should be used when we're in the final state (success or failure).

If the pipeline failed, error_log in the response is the exact error that caused the failure

FIXED.


[FFM-11904]: Improve example and add some documentation.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


why testcase doesn't support 'delete' , only support permanent delection'?

I found the below statements in TFS2018 links:
We only support permanent deletion of test artifacts such as test plans, test suites, test cases, shared steps and shared parameters. Deleted test artifacts won't appear in the recycle bin and cannot be restored. Deletion of test artifacts not only deletes the selected test artifact but also all its associated child items such as child test suites, test points across all configurations, testers (the underlying test case work item doesn't get deleted), test results history, and other associated history.
and our customer are some confused, why testcase can't be deleted and restored, why they only support 'permanent delection'? and the bug/task can support both?

@JunQianZ I apologize for the great delay in addressing your feedback. I have forwarded it to the testing product team people. If you think that we should have a feature added that supports delete/recovery of test artifacts, please file a feature request here.


Docs: contradiction in docs about single instance apps

Describe the bug
There is a contradiction info about the default behavior of app instancing. I think in WinUI / Windows App SDK the correct is that they are multi-instance by default.
Steps to reproduce the bug
The links:

https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/applifecycle/applifecycle-instancing :

WinUI apps are single-instanced by default

https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/applifecycle/applifecycle-single-instance:

WinUI apps are multi-instanced by default.

Expected behavior
No response
Screenshots
No response
NuGet package version
None
Packaging type
No response
Windows version
No response
IDE
No response
Additional context
No response

Hi @fabianoriccardi. Thanks for reporting this inconsistency. I'll verify the behavior again, but in my testing of WinUI apps out of the box, they are multi-instanced. I'll verify and fix this issue in the docs and will follow up when the changes are merged.

The docs have been updated so the first page you've linked says that WinUI apps are multi-instanced by default. I have one minor tweak to the wording from a second PR that hasn't published yet, but it should be live in 10-15 minutes.

Great!

WinUI apps are multi-instanced by default but have the ability to become single-instanced by deciding at launch-time whether to create a new instance or activate an existing instance instead.

I don't remember the exact wording of this whole sentence, but now, for sure, reflects what exactly happens: at the core the app, in the core, is always a multi-instance app, but the second instance can quit itself and reactivate the already existing instance. As a developer feels like a hack, but from user perspective it works perfectly.


ServiceExport deletion leaks targetgroup if the corresponding K8sService have already been deleted

ServiceExport deletion leaks targetgroup if the corresponding K8sService have already been deleted.
More details on how to reproduce this bug (could 100% reproduce it):
For the ServiceExport, it could have 2 scenarios:

Scenario1, all good no bug:

Given: create one k8sService, and one its corresponding ServiceExport, create ServiceExport could trigger controller to create a target group success
Now the k8sService still exists,  if I do kubectl delete serviceexport myserviceexport , could trigger target group deletion correctly

Scenario2, has bug,  target group leaking

Given: create 1 k8sService, and 1 its corresponding ServiceExport, create ServiceExport could trigger controller to create a target group success
do kubectl delete service myservice
Now the k8sService does not exist, if I do kubectl delete serviceexport myserviceexport,  target group deletion will not be triggered, which cause target group leaking

We should use service finalizer to prevent K8S service being deleted,  if this service is referenced by HTTPRoute or ServiceExport

We should use service finalizer to prevent K8S service being deleted, if this service is referenced by HTTPRoute or ServiceExport
Can not quite get what the benefit of it? That can also increase the chance the user is blocked to delete K8S services?

Why would we interrupt k8s service lifecycle though? We already have a behavior of deregistering all targets when service is deleted. HTTPRoute will have a rule that points to empty upstream but it is fine because that's actually the way the user configured k8s config. We should update HTTPRoute resolvedRef status to false instead.
I believe we should move away of user experience of waiting forever. Finalizers are hooks that handle necessary cleanup process, and should not be used for blocking users.

This is very similar to the logic that lattice had on target group and service that you are not allowed to delete rules or services if it references some target group.
Allow deleting K8S service while it is referenced by other objects (e.g. serviceexport, httproute) make those object handling logic more complicated and error prone.

for "user experience of waiting forever.", i believe it is only come from kubectl command kubectl delete xxxx.  If you want a error msg (similar to lattice CLI), you need to add webhook for our controller.

I agree, and webhooks seem like a much better solution than finalizers.

Partially fixed by: https://github.com/aws/aws-application-networking-k8s/pull/389
The rest part of (ServiceExport's ServiceImport refered by Route) bug fix  need the targetGroup datastore refactoring


no parse error when an abstract method contain body

PHP Usually throws a Fatal Error when an abstract method contain a body, even if its not used. but PHP-parser does not.
example :
<?php

namespace Foo;

abstract class Bar extends Baz
{
    private $int = 5;
    protected $val = 'value';
    public $bol = false;

    abstract function name(): string
    {
    }

	public function greet(): void
	{
		echo "Hello World";
	}
}

$main = function (int $argc,string ...$args): void {
    $class = new class extends Bar {
        public function name(): string {
            return 'azjezz';
        }
    };
    $class->greet();
};

$main($_SERVER['argc'], ...$_SERVER['args']);

Expected :
fatal error: Abstract function Foo\Bar::name() cannot contain body in %s on line %d

PHP Behavior :
https://3v4l.org/llHmE

PHP equally parse class and abstract class and checks methods at the compile step.
See zend_language_parser.y:504-519 and zend_compile.c:5780
There are many such checks, so I created another issue #47


Error on Android payload

Here is the log `Traceback (most recent call last):
File "/home/kali/BLE/./keystroke-injectio
n-android-linux.py", line 81, in
client.send_keyboard_report() File "/home/kali/BLE/injector/client.py",
line 119, in send_keyboard_report self.c19.send(keyboard_report(*args))
File "/home/kali/BLE/injector/client.py",
line 32, in send
raise ex
File "/home/kali/BLE/injector/client.py", line 27, in send self.sock.send(data)
File "", line 5, in send bluetooth.btcommon.BluetoothError: [Errno 1
04] Connection reset by peer`
It just stops after that Sorry for the format

I believe your issue is a duplicate of #12, you can see my full reply there, but TL;DR, Android systems after Nov 2018 are accidentally mostly protected from this exploit


Undocumented breaking change from lucide-angular v0.291.0 to v0.292.0

Package

[ ] lucide
[X] lucide-angular
[ ] lucide-flutter
[ ] lucide-preact
[ ] lucide-react
[ ] lucide-react-native
[ ] lucide-solid
[ ] lucide-svelte
[ ] lucide-vue
[ ] lucide-vue-next
[ ] Figma plugin
[ ] source/main
[ ] other/not relevant

Version
v0.291.0
Browser

[X] Chrome/Chromium
[ ] Firefox
[ ] Safari
[ ] Edge
[ ] iOS Safari
[ ] Opera
[ ] Other/not relevant

Operating system

[ ] Windows
[ ] Linux
[ ] macOS
[x] Other/not relevant

Description
The data type of LucideIconData changed in 0.292.0.  This change was not documented.
lucide-angular is deeply integrated in our component library.  This change is technically breaking because it modifies the type of the Input properties on some of our components.
Example property:
  /**
   * {@link https://lucide.dev/icons lucide-angular} icon, positioned after the label.
   * Can also be supplied as a TemplateRef.
   */
  @Input() endIcon?: LucideIconData | TemplateRef<any>

0.291.0
export type LucideIconData = readonly [string, HtmlAttributes, LucideIconNode[]?];
0.292.0
export type LucideIconData = readonly LucideIconNode[];
Are there plans to support semver? I'd like to avoid future breaking changes, if possible.  Thanks!
Steps to reproduce
Inspect the change from 0.291.0 to 0.292.0
0.291.0
https://github.com/lucide-icons/lucide/blob/0.291.0/packages/lucide-angular/src/icons/types/index.ts#L3
0.292.0
https://github.com/lucide-icons/lucide/blob/0.292.0/packages/lucide-angular/src/icons/types/index.ts
Checklist

[X] I have searched if someone has submitted a similar issue before and there weren't any. (Please make sure to also search closed issues, as this issue might already have been resolved.)

Hey @R-Bower,
It was changed in https://github.com/lucide-icons/lucide/pull/1641 as part of us getting ready for Lucide v1 and a larger infrastructure based on the new format - a format that's already used by all our other packages.
But you're absolutely right, we could've added this to release notes. (even if third party usage of this format is not documented either).

I retroactively added this to the release notes: https://github.com/lucide-icons/lucide/releases/tag/0.292.0


Failed to establish a new connection: [WinError 10060]

After implementing changes discussed in issue #32, the script started working again. But an  attempt to resume an unfinished download, which normally should skip already downloaded videos, results in multiple errors.
Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

A workaround was suggested by @mahasatwa to set a ''timeout' parameter for 'requests.get' on line 152.
response = requests.get(dl_url, allow_redirects=True, stream=True, timeout=15)
By setting a 30 seconds timeout, I got past this error and I was able to resume an unfinished
Duplicate of # download.
Originally posted by @mahasatwa in https://github.com/kallqvist/skillshare-downloader/issues/32#issuecomment-662021859


I stopped by and translated it into french

I stopped by and saw there wasn't any french translation of the README.md.
I hope it will fit.

Coverage remained the same at 97.723% when pulling d4a245981467fcc5a6bd5ac2ac9cadc7a269bac6 on GolfRomeoEchoGolf83:master into 5ba6d949c117c4c202596d994f42843304f86ccf on jshint:master.

@GolfRomeoEchoGolf83 can you please sign the CLA? Thanks!

Sorry I didn't see that CLA signature was missing.


Azure SQL : DacpacTask 1.4 fails when {Username}@{ServerName} is specified for the user

It looks like vsts-tasks/Tasks/SqlAzureDacpacDeployment/Utility.ps1 in #3263
was updated to append $serverName IF there is already an @ in the $sqlUsername.  That seems backwards and causes it to fail with:
##[error]Unable to connect to master or target server 'DBNAME'. You must have a user with the same password in master or target server 'DBNAME'
if($sqlUsername)
        {
            if($sqlUsername.Contains('@'))   <== Should that be NOT contains?
            {
                $sqlUsername = $sqlUsername + "@" + $serverName
            }

@muttech
In task input for Azure SQL dacpac task, input for SQL server and username are separate. So username should be provided without @servername . Can you help me is there any reason for providing username in username@servername format. If yes, will surely take it into consideration.
In provided snippet of code, this condition in being added for reason following reason -
if username itself contains @ symbol. E.g. username is - sql@dmin.  To handle such case we have to explicitly add @servername.  So we have to pass sql@dmin@servername to sqlpackage.exe
 if($sqlUsername.Contains('@'))                {                 $sqlUsername = $sqlUsername + "@" + $serverName              }
Since your point is also a valid point one can provide username@servername and taskshould not fail at this condition , will add that condition too in next sprint.

@Ajay-MS the variable for username in the release was configured that way many months ago (I honestly don't remember if there was prior documentation showing that, or I just added the @servername to it - although I don't know why I would have decided to do so without seeing it elsewhere) - and it worked until mid-day yesterday, which led to the discovery of the task changing from 1.2 to 1.4 and that change being introduced.
I honestly had no idea that you could create a username with a @ in it, so that's a more interesting use-case to me.
Thanks for the explanation!


DOC-1607: Correct RN for #64004

Fixes DOC-1607

This change is

@jordanlewis Just touching base again. Are you able to review this for me?

Hi, sorry I've been on PTO returning next week. It looks good to me, thanks!

I'm so sorry for bothering you. Should have checked Slack first. Thank you!

@gemma-shay TFTR!


Provide Kubeflow XGBoostJob support for MultiKueue

What type of PR is this?
/kind feature

What this PR does / why we need it:
The PR introduces a new MultiKueue adapter to handle XGBoostJob (Kubeflow).
We want to extend MultiKueue capabilities to satisfy the needs of early adopters.
Which issue(s) this PR fixes:

Relates #2552
Special notes for your reviewer:
Does this PR introduce a user-facing change?

MultiKueue: Support for the Kubeflow XGBoostJob

/ok-to-test

/lgtm
/approve
Thanks!


3.0.0-alpha5: New dependency IPC::Cmd in util/perl/OpenSSL/config.pm

Commit a3310b182ca3952c2876792b35035b54a4c77713 introduced a new dependency on IPC::Cmd in config.pm. It is triggered by "use OpenSSL::config;" in Configure line 23.
On RHEL 7 the perl-IPC-Cmd rpm unfortunately has a few dependencies that are unwanted, like gdbm-devel and libdb-devel. Therefore it would be good to not rely on IPC::Cmd - or document its need in NOTES-Perl.md.
The full error is:
Can't locate IPC/Cmd.pm in @INC (@INC contains: /path/to/util/perl /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . /path/to/external/perl/Text-Template-1.56/lib) at /path/to/util/perl/OpenSSL/config.pm line 18.
BEGIN failed--compilation aborted at /path/to/util/perl/OpenSSL/config.pm line 18.
Compilation failed in require at /path/to/Configure line 23.
BEGIN failed--compilation aborted at /path/to/Configure line 23.
The calling file Configure also uses IPC::Cmd but has an eval-require construct to check for its existence and fallback code for the case it is missing. Maybe one can use the same approach in config.pm, or even move the code to some common place and use it in Configure and config.pm.

Ping @levitte

According to corelist, IPC::Cmd has been around as a core perl module since v5.9.5.  That's within our requirements.  I was also careful to check that the function we use (IPC::Cmd::can_run) has been around long enough (I can at least trace it back to perl v5.10.1).
Why RHEL makes perl-IPC-Cmd on devel packages, I cannot answer.  I could understand that as build dependencies, but not for run-time package dependencies.
This sounds like a packaging problem more than a problem in our usage.  I can't see how our using it within our build parameters is a bug, sorry.
Maybe you could ask RH for an explanation on that (odd) package dependency?

Some history is in the following ticket:
https://bugzilla.redhat.com/show_bug.cgi?id=1129443
IPC::Cmd has a dependency on ExtUtils::MakeMaker which in turn needs perl-devel. That one includes header files which which include other headers from gdbm and db2.
In RHEL 8 they solved the issue by slightly adjusting IPC::Cmd: the onyl feature form MakeMaker that was used by IPC::Cmd was MM->maybe_command(). RHEL noticed this function was pretty simple and introduced a new module in RHEL 8, ExtUtils::MM::Utils. It contains the following comment:
These methods are copied from LExtUtils::MM_Unix. Other operating systems
are not supported yet. The reason is this
L<a hack for Linux
distributions|https://bugzilla.redhat.com/show_bug.cgi?id=1129443>.
And the code is:
if (!exists $INC{'ExtUtils/MM.pm'}) {
*MM::maybe_command = *ExtUtils::MM::maybe_command = &maybe_command;
}
sub maybe_command {
my($self,$file) = @_;
return $file if -x $file && ! -d $file;
return;
}
1;
It is fine for me to apply this fix locally to my RHEL 7 build environment for OpenSSL. At least the situation is well-documented now. So if you like, you might close this issue. Thanks for looking at it!

Nice investigation!  I'm glad you could fix it locally.


Test issue for the new workflow

works :) (not sure why it works now but didnt on the PR but whatever)


feature: CE-554

Description
Please provide a summary of the change and the issue fixed. Please include relevant context. List dependency changes.
Fixes # (issue)
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist

[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Further comments

Thanks for the PR!
Any successful deployments (not always required) will be available below.
Backend available
Frontend available
Once merged, code will be promoted and handed off to following workflow run.
Main Merge Workflow

@cnesmithsalus I saw a couple of UX items that I think we should adjust (unless you were told to explicitly do any of these:

If only the species is entered we should drop the comma at the end as it looks silly
If there is no outcome entered we should hide the officer initial placeholder
the required fields show as soon as you select an outcome (before pressing save)

@cnesmithsalus - Sorry found another issue :(

Expand Drugs
Fill in Everything except the officer and date
Press Save

This should not let me save, but it does.

@afwilcox I want to throw my computer out of the window -- I think it should be good to go now


Fix AppleScript issue

Found it on this video by @trevordmiller:

It says:

osascript: /Users/trevormiller/Google: No such file or directory

We should probably mute any warnings from osascript (since it’s optional anyway) but it would be nice to figure out what exactly failed here and why.
Relevant code.

@gaearon Is this something I did wrong? Or do you need me to run any commands to send you logs or something?

You did nothing wrong, our AppleScript command to launch Google Chrome just failed for some reason. Maybe it always fails when Chrome is not open (need to check).

@gaearon Thanks for checking out my video!

Thanks for recording! Would be cool to make a few more. Topic ideas:

Publishing to GitHub Pages
Displaying Lint Output in the Editor
Adding React Bootstrap

@gaearon For sure! Would you mind if I recorded these or would you like to do it?

Sure, go ahead, I can’t find the time now. 😄

@gaearon Sounds good. Also, maybe "Extending react-scripts" to basically show how to add things like tests with npm scripts and &&; something like:
package.json
...
"scripts": {
  "test": "ava"
  "start": "npm-run-all npm run test -- -w && react-scripts start"
  "prebuild": "npm run test"
}
...

Do you think that would be helpful or would you recommend another approach?
Sorry I realize this is off-topic of this issue.

@trevordmiller This would be fine except you’ll probably want to explain how to do this on Windows too. This tool supports Windows (and we care about this).

@gaearon Cool :)


test fails for urls without tld (http://localhost)

Describe the bug
When the input url doesn't end with a tld, it is treated as html and all tests will fail.
To Reproduce

Windows, Xampp, Apache, test website is in /htdocs/mysite
run apache, website is available under http://localhost/mysite
data test is in /htdocs/datatest

cd /path/to/htdocs/datatest
npm init
npm install structured-data-testing-tool

Now I tried an example snippet from README.md

/datatest/index.js:
const { structuredDataTest } = require('structured-data-testing-tool');
const { Twitter, Facebook } = require('structured-data-testing-tool/presets');

const url = 'http://localhost/mysite';

structuredDataTest(url, { presets: [ Twitter, Facebook ] })
.then(res => {
  console.log("All tests passed.");
})
.catch(err => {
  if (err.type === 'VALIDATION_FAILED') {
    err.res.failed.forEach(test => {
      console.error(test);
    })
  } else {
    console.log(err);
  }
});

run node index.js --> all tests failed

If I use the html output of my site as input string, everything works like expected.
Solution 1
When I write my test, I know, that I always expect a url, so I can skip that check by calling structuredDataTestUrl directly.
const { structuredDataTestUrl } = require('structured-data-testing-tool');
const url = 'http://localhost/mysite';
structuredDataTestUrl (url, {/* ... */}).then(/* ... */).catch(/* ... */);

Solution 2
Add options to isURL check to ignore tld:
https://github.com/glitchdigital/structured-data-testing-tool/blob/master/index.js#L434
if (validator.isURL(input, { require_tld: false )) {/* ... */}

And thanks for your tool. Now, that I found my issue, I can fiddle and explore some more of it's features.

Thank you @raffaelj, this was an absolutely brilliant bug report!
I really appreciate the time researching the fix and the example code to replicate the issue.
(* Sorry about all those lines of code in one file - I wrote it and even I struggle with figuring out what is going on there when I haven't looked at it in a few weeks.)
I was able to use the report to write a test, confirm I could replicate the issue, then look up the option suggested and confirm the proposed fix resolved the bug and just published the fix as v3.6.

Thanks for the flowers and for the fast fix. I'm glad I was able to help.

Sorry about all those lines of code in one file - I wrote it and even I struggle with figuring out what is going on there when I haven't looked at it in a few weeks.

To be honest: Your code is a mess. I needed four text editor instances on two monitors to track your code down when I tried to reproduce your validation mechanisms. So I rewrote your tool and the web-auto-extractor into a more readable style. I'll open a new issue for this topic with feedbask soon.


Update Notification

What's the Problem?
Not notified when a new version is available
Solution/Idea
Add a setting which when enabled, will notify the user when an update is available to download.
Alternatives
Not having a setting and Files will notify the user of an update
Priorities
No response
Files Version
No response
Windows Version
No response
Comments
No response

We need a little more information from you in order to investigate this issue, please reopen this issue once you have filled out the issue template.


i like youtube center a lot but i have one virus issue.

when i enable youtube center extension there is a virus named "ads not by this site" appears in my browser.and when i disable it it doesn't appear.
so what to  do to use youtube center safely without that virus?
note:
i also tried to use anti malware to remove it but it doesn't help.

It's definitely not youtube center's fault. Your browser was probably hijacked by a rogue extension and this is why it happens.


running tcmu-runner

I was just beginning to contact tcmu-runner and installed according to README.md;when I want to run tcmu-runner:
[root@198 tcmu-runner]# tcmu-runner
2018-04-23 22:18:58.952 14583 [INFO] dyn_config_start:416: Inotify is watching "/etc/tcmu/tcmu.conf", wd: 1, mask: IN_ALL_EVENTS
2018-04-23 22:18:58.987 14583 [ERROR] set_genl_features:281: Could not set features. Error -10
Using targetcli to view backstores does not appear like readme：
/backstores> ls
o- backstores .......................................................... [...]
o- user:glfs .......................................... [Storage Objects: 0]
o- user:qcow .......................................... [Storage Objects: 0]
o- user:rbd ........................................... [Storage Objects: 0]
o- user:file .......................................... [Storage Objects: 0]
o- user:zbc ........................................... [Storage Objects: 0]
But like this：
/backstores> ls
o- backstores ....................................................................... [...]
o- block ........................................................... [Storage Objects: 1]
| o- lun0 .................................... [/dev/sdb1 (0 bytes) write-thru activated]
|   o- alua ............................................................ [ALUA Groups: 1]
|     o- default_tg_pt_gp ................................ [ALUA state: Active/optimized]
o- fileio .......................................................... [Storage Objects: 0]
o- pscsi ........................................................... [Storage Objects: 0]
o- ramdisk ......................................................... [Storage Objects: 0]
/backstores>
Readme said By default, tcmu-runner installs the file, zbc, glfs, qcow and rbd tcmu-runner handlers; I don't know where the problem is now, and hope someone can help me out.

Hi,
It will help to provide some information regarding the test environment.
That said, you may need to copy the handlers to "/lib64/tcmu-runner"
directory for them to show up in targetcli. Look for "handler_*.so" in
tcmu-runner directory after build.
Also keep in mind that "targetcli" you are using may or may not work well
with tcmu devices. In my own experience, I had better luck with recent
kernel + targetcli-fb (as described in README).
Thanks,
Raghu
On Mon, Apr 23, 2018 at 10:33 PM, liny<EMAIL_ADDRESS>wrote:

I was just beginning to contact tcmu-runner and installed according to
README.md;when I want to run tcmu-runner:
[root@198 tcmu-runner]# tcmu-runner 2018-04-23 22:18:58.952 14583 [INFO]
dyn_config_start:416: Inotify is watching "/etc/tcmu/tcmu.conf", wd: 1,
mask: IN_ALL_EVENTS 2018-04-23 22:18:58.987 14583 [ERROR]
set_genl_features:281: Could not set features. Error -10
Using targetcli to view backstores does not appear like readme：
/backstores> ls
o- backstores ..........................................................
[...]
o- user:glfs .......................................... [Storage Objects:
0]
o- user:qcow .......................................... [Storage Objects:
0]
o- user:rbd ........................................... [Storage Objects:
0]
o- user:file .......................................... [Storage Objects:
0]
o- user:zbc ........................................... [Storage Objects:
0]
But like this：
/backstores> ls o- backstores
.......................................................................
[...] o- block ...........................................................
[Storage Objects: 1] | o- lun0 ....................................
[/dev/sdb1 (0 bytes) write-thru activated] | o- alua
............................................................ [ALUA Groups:
1] | o- default_tg_pt_gp ................................ [ALUA state:
Active/optimized] o- fileio
.......................................................... [Storage
Objects: 0] o- pscsi
........................................................... [Storage
Objects: 0] o- ramdisk
......................................................... [Storage Objects:
0] /backstores>
Readme said By default, tcmu-runner installs the file, zbc, glfs, qcow and
rbd tcmu-runner handlers; I don't know where the problem is now, and hope
someone can help me out.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/open-iscsi/tcmu-runner/issues/404, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACr4-30C5Pqvl8jGFxXDZEMMHb18dSJtks5tro7kgaJpZM4Tg6Io
.

targetcli-fb

Also make sure you are using rtslib-fb. You can get both from https://github.com/open-iscsi/.

@mikechristie @draghuram thanks,Can see user:rbd. . .
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 1]
| | o- lun0 ..................... [/dev/sdb1 (0 bytes) write-thru deactivated]
| |   o- alua ............................................... [ALUA Groups: 1]
| |     o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
| o- user:glfs .......................................... [Storage Objects: 0]
| o- user:qcow .......................................... [Storage Objects: 0]
| o- user:rbd ........................................... [Storage Objects: 1]
| | o- rbd0 ................................... [pool/rbd1 (1.0GiB) activated]
| |   o- alua ............................................... [ALUA Groups: 0]
| o- user:zbc ........................................... [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.198.x8664:sn.226f01c32bd5 ......... [TPGs: 1]
|   o- tpg1 ........................................... [no-gen-acls, no-auth]
|     o- acls ...................................................... [ACLs: 1]
|     | o- iqn.2003-01.org.linux-iscsi.198.x8664:sn.226f01c32bd5:client  [Mapped LUNs: 1]
|     |   o- mapped_lun0 ............................... [lun0 user/rbd0 (rw)]
|     o- luns ...................................................... [LUNs: 1]
|     | o- lun0 ........................................... [user/rbd0 (None)]
|     o- portals ................................................ [Portals: 1]
|       o- xx.xx.xx.xx:3260 ........................................... [OK]
o- loopback ..................................................... [Targets: 0]
But it looks like I have a new problem. I created an rbd0 in user:rbd and then created iqn, lun0 in iscsi. The iscsi client can also discover this iqn and the login is successful; however, it is not possible to mount rbd0 in user:rbd on the client.
I see o- lun0 ........................................... [ User/rbd0 (None)]. After user/rbd0 does not know why is none.
Or is the rbd0 created by user:rbd need to be mounted on the client and need additional operations?

| | o- rbd0 ................................... [pool/rbd1 (1.0GiB) activated]
| | o- alua ............................................... [ALUA Groups: 0]
It may due to your alua/ without any alua entity.
It should something like:
| o- user:glfs .............................................................................................. [Storage Objects: 4]
| | o- block01<EMAIL_ADDRESS>(1.0GiB) activated]
| | | o- alua ................................................................................................... [ALUA Groups: 2]
| | |   o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| | |   o- glfs_tg_pt_gp .......................................................................... [ALUA state: Active/optimized]
What's you kernel version ?
Or could you provide the tcmu-runner or kernel's logs when you are trying to login or doing the mounting  ? Any errors/warning ？

@lxbsz thanks,
Linux 198 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@198 tcmu-runner]# tcmu-runner --handler-path /usr/local/lib64/tcmu-runner
2018-04-24 20:19:32.142 15900 [INFO] dyn_config_start:416: Inotify is watching "/etc/tcmu/tcmu.conf", wd: 1, mask: IN_ALL_EVENTS
2018-04-24 20:19:32.181 15900 [ERROR] set_genl_features:281: Could not set features. Error -10
2018-04-24 20:19:32.182 15900 [WARN] tcmu_block_device:395 rbd/rbd0: Kernel does not support the block_dev action.
2018-04-24 20:19:32.182 15900 [WARN] timer_check_and_set_def:391 rbd/rbd0: osd op timeout ((null)) must be larger than osd heartbeat grace (20) + interval (6)!
2018-04-24 20:19:32.182 15900 [WARN] timer_check_and_set_def:410 rbd/rbd0: Will set the osd op timeout to 31 instead!
no monitors specified to connect to.
2018-04-24 20:19:32.199 15900 [ERROR] tcmu_rbd_image_open:447 rbd/rbd0: Could not connect to cluster. (Err -2)
2018-04-24 20:19:37.202 15900 [ERROR] add_device:568: handler open failed for uio0
2018-04-24 20:19:37.202 15900 [WARN] tcmu_unblock_device:418 rbd/rbd0: Kernel does not support the block_dev action.
Before I created a lun0 in the backstores/block that there alua, then I successfully mounted on the client.
I just created a user:rbd on the readme;
I saw some needs on the Internet like this: rbd create ceph-client1-rbd1 --size 10240;rbd map...
The rbd0 path I created is pool/rbd1. Is it something I need to map this rbd to something like /dev/sdc so that the client can mount this /dev/sdc?

2018-04-24 20:19:32.199 15900 [ERROR] tcmu_rbd_image_open:447 rbd/rbd0: Could not connect to cluster. (Err -2)
2018-04-24 20:19:37.202 15900 [ERROR] add_device:568: handler open failed for uio0
Have you resolved this errors ?

@lxbsz
Sorry, no. I do not know how to solve this problem.
I now want to simply run the tcmu-runner example, I did not deploy the cluster. I see that there are ceph and gluster options on the readme. What do I need to do now to mount?

@lxbsz
If I want to create user:glfs like the one you give, do I need to choose gluster?

@lxbsz
I git clone gluster/gluster-block
[root@198 sysconfig]# systemctl restart gluster-blockd
Failed to restart gluster-blockd.service: Unit not found.

NO, if so you no need to install gluster or ceph, you can just use the file_example handler to have a try.

@lxbsz
How to use file_example? Can also create user:rbd or user:glfs and mount it?

Make sure /lib64/tcmu-runner/ has the handler_file.so handler, or you can just copy it there:
[root@gblock2 rhel7]# ls /lib64/tcmu-runner/
handler_file_optical.so  handler_file.so  handler_file_zbc.so  handler_glfs.so  handler_qcow.so  handler_rbd.so  libtcmu.so
[root@gblock2 rhel7]#

$ targetcli /backstores/user:file create myfile 1G myfile
$ targetcli ls
....
  | o- user:file .............................................................................................. [Storage Objects: 1]
  | | o- myfile ...................................................................................... [myfile (1.0GiB) deactivated]
  | |   o- alua ................................................................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]

Then create the iscsi LUN as needed.

@lxbsz ,thanks a lot.
$ targetcli /backstores/user:file create myfile 1G myfile
Is the second myfile a real file (img) or /dev/sdb?

@lxbsz
After luns was created fileio appeared alua
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 1]
| | o- myfile ........................ [myfile (0 bytes) write-thru activated]
| |   o- alua ............................................... [ALUA Groups: 1]
| |     o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
| o- user:fbo ........................................... [Storage Objects: 0]
| o- user:file .......................................... [Storage Objects: 1]
| | o- myfile .................................. [myfile (2.0GiB) deactivated]
| |   o- alua ............................................... [ALUA Groups: 0]
| o- user:glfs .......................................... [Storage Objects: 0]
| o- user:qcow .......................................... [Storage Objects: 0]
| o- user:rbd ........................................... [Storage Objects: 0]
| o- user:zbc ........................................... [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.198.x8664:sn.2247f71c798c ......... [TPGs: 1]
|   o- tpg1 ........................................... [no-gen-acls, no-auth]
|     o- acls ...................................................... [ACLs: 1]
|     | o- iqn.2003-01.org.linux-iscsi.198.x8664:sn.2247f71c798c:client  [Mapped LUNs: 1]
|     |   o- mapped_lun0 ........................... [lun0 fileio/myfile (rw)]
|     o- luns ...................................................... [LUNs: 1]
|     | o- lun0 .................. [fileio/myfile (myfile) (default_tg_pt_gp)]
|     o- portals ................................................ [Portals: 1]
|       o- <IP_ADDRESS>:3260 ........................................... [OK]
o- loopback ..................................................... [Targets: 0]
Clients：
[cp@localhost usr]$ lsscsi
[2:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sda
[4:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sdb
[21:0:0:0]   disk    LIO-ORG  FILEIO                4.0   /dev/sdc
[cp@localhost usr]$ sudo mount /dev/sdc /home/cp/mountdir
mount: you must specify the filesystem type
[cp@localhost usr]$ sudo mkfs.ext4 /dev/sdc
mke2fs 1.41.12 (17-May-2010)
/dev/sdc is entire device, not just one partition!
Proceed anyway? (y,n) y
mkfs.ext4: Device size reported to be zero.  Invalid partition specified, or
partition table wasn't reread after running fdisk, due to
a modified partition being busy and in use.  You may need to reboot
to re-read your partition table.
Is it possible that myfile created in user:file cannot be formatted and mounted?

@linyuan870
From you logs, it seems work for you, right ?
Yeah, it possible, if there has something wrong with your configuration or something others.

@lxbsz thanks very much,
It looks like there is only one last step left.
Based on your experience.Whether the error is on the server or the client？

@linyuan870 Both sides are possible.

@lxbsz， OK
Let me see for myself where the problem lies. Thank you very much for spending so much time.

@liny You need to create the backstore for file in a certain way. One
required parameter is "size". Here is one example:
"/backstores/user:file create cfgstring=//tmp/testfile name=test
size=1057964544"
On Wed, Apr 25, 2018 at 5:15 AM, liny<EMAIL_ADDRESS>wrote:

@lxbsz https://github.com/lxbsz， OK
Let me see for myself where the problem lies. Thank you very much for
spending so much time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-iscsi/tcmu-runner/issues/404#issuecomment-384219634,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACr4-zjxW_S0HEDtnsIZ6feX4LM_hs-4ks5tsD69gaJpZM4Tg6Io
.

@draghuram thanks,
But it seems to be the same
$ targetcli /backstores/user:file create myfile 1G myfile
client:
[cp@localhost tmp]$ lsscsi
[2:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sda
[4:0:0:0]    disk    ATA      VBOX HARDDISK    1.0   /dev/sdb
[23:0:0:0]   disk    LIO-ORG  FILEIO           4.0   /dev/sdc
I am on the server side，view to testfile：
[root@198 tmp]# ll -t
total 20
-rw-------. 1 root root    0 Apr 25 05:49 testfile
I found | | o- test .......................... [/tmp/testfile (1009.0MiB) deactivated
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 1]
| | o- tmp-testfile ........... [/tmp/testfile (0 bytes) write-thru activated]
| |   o- alua ............................................... [ALUA Groups: 1]
| |     o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
| o- user:fbo ........................................... [Storage Objects: 0]
| o- user:file .......................................... [Storage Objects: 1]
| | o- test .......................... [/tmp/testfile (1009.0MiB) deactivated]
| |   o- alua ............................................... [ALUA Groups: 0]
| o- user:glfs .......................................... [Storage Objects: 0]

handler_file just does read()/writes() to any old file like /home/mikechristie/myfile. You do not need to do anything special. It works just like lio's target_core_file backend.

@mikechristie thanks,
I think I shouldn't say it clearly. I wanted to be able to mount a cluster file system like gluster on the client (like gluster-block). Then this cluster file system is its own.

@mikechristie
Not a simple example of running.

Hi, I am not entirely sure if I understand your question completely but I
will take a stab at it any way. With tcmu-runner, you can implement LUNs
that can act as devices. File systems are at a different level so if you
want to implement a file system, I don't think that can be done using TCMU
plugin.
Thanks,
Raghu
On Fri, Apr 27, 2018 at 5:21 AM, liny<EMAIL_ADDRESS>wrote:

@draghuram https://github.com/draghuram @lxbsz
https://github.com/lxbsz @mikechristie https://github.com/mikechristie
thanks,I finally compiled a latest kernel successfully mounted out.
I finally want to ask a question. If I want to write my own file system
through tcmu-runner so that the client can mount it, it is necessary to
compile the file system into something like handler_file.so and copy it to
/lib64/tcmu-runner/? The cluster is also its own. Can you give me some
guidance? Thank you very much.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-iscsi/tcmu-runner/issues/404#issuecomment-384915775,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACr4-xh_i4CfDgqmy05vhW-cUNFpL9Mdks5tsuMmgaJpZM4Tg6Io
.

@draghuram
Such as using targetcli cd backstores, can find my implementation of user:myfile. After a series of operations, the client mounts my cluster storage.

@mikechristie Hi
If I want to write a makefile to compile tcmu-runner through g++, will the changes work?

I do not know enough about g++. I have no idea.

@mikechristie OK,thanks


Job Messages Prototypes

[not to be submitted, just for prototyping feedback]

@bjchambers will be toying around with prototype APIs here for emitting job messages.


Add NodeJS components

Please answer these questions before submitting pull request

Why submit this pull request?
[ ] Bug fix
[x] New feature provided
[ ] Improve performance

Coverage increased (+0.004%) to 21.623% when pulling 850709f5c4b7e538541058d7ba8d323c221ed230 on ascrutae:feature/nodejs-component into 01b5d4901e3a894f486bd4980dbba8c9f282dfd4 on apache:master.


Preserving unloaded related data

Describe the bug
Sorry, here I am again.
Repro here: https://dotnetfiddle.net/wDbM8i
Parent is being saved with some stuff changed but with
The parent has a one-to-many to details but when saving the parent without details loaded I get an error:
The association between entity types 'Parent' and 'Detail' has been severed, but the relationship is either marked as required or is implicitly required because the foreign key is not nullable. If the dependent/child entity should be deleted when a required relationship is severed, configure the relationship to use cascade deletes. Consider using 'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the key values.
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Program.Main()
Command terminated by signal 6

Expected behavior
Expectation is to save everything, and leave all the associated details as is.
Additional context
I think my fiddle is using 5.0.21 but I ran into this with 5.0.22. It might be that Detached-PatchTypes is the solution to this, but I'm using newtonsoft json for serializing requests so I couldn't get that to work in my project.

Hi again,
Yes, it is the "undefined" issue that comes with C#, null is a value, and will be saved as null, overriding any previous existing value.
And yes again, PatchTypes is one possible solution... I was sure newtonsoft was already deprecated, but will try to make it work and let you know.
Another posibility would be to use more DTOs, and just omit the values you don't want to map. GraphQL papers recommend to have 1 class per command, and usually calls it [Command]Input... for example SaveUserInput. This can be applied to Rest too.
Go to your fiddle and delete "Parent" after the "new" keyword, it will become an anonymous class and the mapping will work.
context.Map(new Parent{Id = 1...
context.Map(new {Id = 1...
Thanks for your feedback, as you help me to improve the library.
It's only me here, and I have few time, no help and negative money :) so if you like the library I'd be grateful if you mention it to your colleagues, maybe we can involve more people.
I'll let you know what happens with newtonsoft.

I was sure newtonsoft was already deprecated
MS is trying to make it so... But google a bit, everyone is still using it seems to me.

But yeah, you're right, I should make proper viewmodels.

Thanks for your feedback, as you help me to improve the library.
Thank you for responding that quick! For a one man maintainer you're doing terrific!

maybe we can involve more people
I'm trying to wrap my head around it, but so far its a bit much

I'll let you know what happens with newtonsoft.
Thanks a ton!

Not sure if you saw it in the other project, but already implemented newtonsoft converter for patch types


Stuttering every 30 seconds on Sonos One

Attached system information from Sonos app and log from the app:
Configuration { server_port: Some(5901), auto_resume: false, sound_source: "Speakers (Realtek High Definition Audio)", sound_source_index: None, log_level: Info, ssdp_interval_mins: 1.0, auto_reconnect: true, disable_chunked: false, use_wave_format: true, bits_per_sample: Some(24), streaming_format: Some(Wav), monitor_rms: false, capture_timeout: Some(2000), inject_silence: Some(false), last_renderer: "Sonos One Living Room - Sonos One Media Renderer", last_network: "<IP_ADDRESS>", config_dir: "<removed>", config_id: Some("") }
Setup audio sources
Now running at ABOVE_NORMAL_PRIORITY_CLASS
Capturing audio from: Speakers (Realtek High Definition Audio)
Default audio SupportedStreamConfig { channels: 2, sample_rate: SampleRate(48000), buffer_size: Range { min: 0, max:<PHONE_NUMBER> }, sample_format: F32 }
Discover networks
Starting SSDP discovery
The streaming server is listening on http://<IP_ADDRESS>:5901/stream/swyh.wav
Streaming sample rate: 48000, bits per sample: 24, format: WAV
The wave_reader is now receiving samples
AV Start playing on Sonos One - Sonos One Media Renderer host=<IP_ADDRESS> port=1400 from <IP_ADDRESS> using AvTransport Play
AV Stop playing on Sonos One - Sonos One Media Renderer host=<IP_ADDRESS> port=1400
Received request /stream/swyh.wav from <IP_ADDRESS>:34560
Streaming audio/wave;codec=1 (WAV), input sample format F32, channels=2, rate=48000, disable chunked=false to <IP_ADDRESS>:34560

Windows 10 Home v10.0.19044 Build 19044
Let me know if you need anything else.

Could you try to

set the ssdp interval to 10 minutes
disable chunked transfer encoding

and see if it changes anything?
Also older Sonos devices (Play 5 gen 1) had very limited buffer space and are very sensitive to network delays (wifi).

No change unfortunately.
My current config:
[Configuration]
ServerPort = 5901
AutoResume = true
SoundCard = "Speakers (Realtek High Definition Audio)"
LogLevel = "INFO"
SSDPIntervalMins = 10.0
AutoReconnect = true
DisableChunked = true
UseWaveFormat = true
BitsPerSample = 24
StreamingFormat = "Wav"
MonitorRms = false
CaptureTimeout = 2000
InjectSilence = false
LastRenderer = "Sonos One Sonos One - Sonos One Media Renderer"
LastNetwork = "<IP_ADDRESS>"
ConfigDir = "<removed>"
ConfigId = ""

Could you try 16 bits per sample? If it reduces

No change. Still 30 sec.

I've closed everything except the Spotify app. There are no spikes in Task Manager, the CPU is under 10%.
I've attached the log - Log Level: Debug, Autoresume and Autoreconnect disabled.
log.txt

I've tried with vb-audio hifi. There's no change. I also made a recording so you can see what I mean by stuttering (0:14, 0:44, 1:14). I'll try from a different computer on the same network and report back.

The exactly 30 seconds interval is strange. Perhaps the Sonos has a problem with the audio format (wav/lpcm).
It's a shame that Sonos does not do FLAC (AFAIK).

I've connected the Sonos via Ethernet directly to the router and lo and behold the stuttering is gone - also with autoresume, autoreconnect and 24bit enabled. I have on idea why, especially considering that I have a pretty decent router and the speaker was 2 meters away in the same room.

Good to hear that the stuttering is gone with an ethernet connection, but still wifi should not cause stuttering...

I've discovered something very intriguing. When I scan the network with the Fing app for Android, playback (on wifi) stutters badly (even stops for a few seconds) while the phone is scanning for other devices connected to the network.

Apparently Sonos can have issues with wifi: https://en.community.sonos.com/speakers-229128/troubleshooting-sonos-on-wifi-6856334. I guess the root of the problem is that they do not have a configurable buffering (latency) setting to cope with network latency.

I moved the speaker from my 2.4ghz network to the 5ghz one and it works perfectly now. No more stutters every 30 seconds, in fact no more stutters at all. I can even scan the network with Fing and nothing happens. Does this make any sense?

It does, as you are probably avoiding interference on the crowded 2.4 ghz band that has a much lower throughput anyway.

Thanks for your time helping me debug my problem and great job on the app BTW! Things would be a lot easier if only my Sonos would support standard Bluetooth or have a headphone jack. I guess it's by design because without paid subscriptions my speaker is pretty much a paper weight.

One last question (because I don't have access to a Sonos): did you ever try flac?

Nothing happens when I select FLAC or LPCM.

Thanks for confirming what I suspected. Flac is not supported when using dlna/upnp.


Local testing

Figure out a way to host a CC application as a test app in fusion framework..
Possible solutions: [Yalc, pnpm link.]

The usage of integration tests has relieved some of the workload here

 Atle made this, might be useful in mocking a portal around a cc app


Increase memory in longrun pipeline

Purpose

To-do

Content

[ ] I have read and checked the items on the review checklist.

bors r+


MSID: 88824 Version: 1 DOI: 2<IP_ADDRESS>0181

"msas": "Neuroscience"
"msid": "88824"
"preprintDoi": "10.1101/2<IP_ADDRESS>0181"
"Reviewed Preprint posted": "2023-08-08"
"Sent for peer review": "2023-04-24"
"Posted to bioRxiv": "2023-04-25" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>0181v3/")
[PLACE PDF URL HERE WHEN AVAILABLE] See step 7
Step 1. Inform bioRxiv
Who can help: @QueenKraken, @nlisgo, @scottaubrey

[x] Manuscript is in data hub index (https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v1/by-publisher/elife/get-by-manuscript-id?manuscript_id=88824)

Step 2. Create preview of manuscript
Who can help: @fred-atherden, @nlisgo, @scottaubrey

[ ] Manuscript is available for preview (https://prod--epp.elifesciences.org/preview/88824/v1)

[ ] Trigger import for staging: https://prod--epp.elifesciences.org/import
[ ] Sync data folder with s3://prod-elife-epp-data/data
[ ] Trigger import for staging: https://staging--epp.elifesciences.org/import
[ ] Sync data folder with s3://staging-elife-epp-data/data
[ ] Pull request created to enhanced-preprint-data
[ ] Post link to pull request below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-manuscript
Step 3: Awaiting public reviews and QC
Who can help: Production team

[ ] QC passed or QC not needed

Who can help: Editorial team

[x] Public reviews available on sciety (https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>0181)
[x] Manuscript and editor details at the top of this issue (Supplied by Editorial team)
[ ] Reviews returned by EPP - https://prod--epp.elifesciences.org/api/reviewed-preprints/88824/v1/reviews

Example
"msas": "Genetics and Genomics", "Neuroscience"
"msid": "84628"
"version": "1"
"preprintDoi": "10.1101/2<IP_ADDRESS>4241"
"articleType": "Reviewed Preprint"
"status": "Published from the original preprint after peer review and assessment by eLife."

"Reviewed Preprint posted": "2023-01-02"
"Sent for peer review": "2022-10-28"
"Posted to bioRxiv": "2022-11-21" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>4241v1")

Editors:
Reviewing Editor
Michael B Eisen
University of California, Berkeley, United States
Senior Editor
Michael B Eisen
University of California, Berkeley, United States

Step 4: Modify manuscripts.json (no PDF)

[ ] Reviewed preprint is published to EPP (https://elifesciences.org/reviewed-preprints/88824)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[ ] Request that a doi be registered

[ ] Post link to #enhanced-preprint comment below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]

Instructions to modify manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/publish-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/334/files
Once the pull request is merged in it should be available a few minutes later.

Request that a doi
Post the following in #enhanced-preprint:
@Fred can you register a doi for https://elifesciences.org/reviewed-preprints/88824

Step 5: Awaiting search reindex

[ ] Reviewed preprint is avaliable on journal homepage (https://elifesciences.org)

The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.

Additional info
If needed, the jenkins pipeline to reindex search can be triggered sooner.
https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 6: Published! Request PDF generation

[ ] Post the link to the PDF url at the top of the issue

[ ] Post link to #enhanced-preprint comment below

#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]
Post the following to the #enhanced-preprint on slack:
@Ryan Dix-Peek please can you generate a PDF for https://elifesciences.org/reviewed-preprints/88824

Step 7: Introduce PDF to data folder and git repo

[ ] PDF is avaliable at https://github.com/elifesciences/enhanced-preprints-data/raw/master/data/88824/v1/88824-v1.pdf

Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-pdf
Step 8: Add PDF url to manuscripts.json

[ ] Reviewed preprint PDF is available for download (https://prod--epp.elifesciences.org/reviewed-preprints/88824)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[PLACE LINK TO PULL REQUEST HERE]

Instructions to add PDF url to manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/add-pdf-url-to-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/397/files
Once the pull request is merged in it should be available a few minutes later.

Step 9: Done!

[ ] Kettle is on!

We have version 2 loaded instead of version 3

This looks pretty odd, but not nothing I can do about it. The equation images are rendered a bit small.

Also the list indicators - some of them should be lower alpha characters. Should hopefully be fixed once other types of list are supported in EPP.
Going to proceed anyway.


[GHSA-g6w6-r76c-28j7] Incorrect Authorization in NATS nats-server

Updates

Affected products

Comments
Make affected products versions SEMVER compliant

Hi there @philpennock! A community member has suggested an improvement to your security advisory. If approved, this change will affect the global advisory listed at github.com/advisories. It will not affect the version listed in your project repository.
This change will be reviewed by our highly-trained Security Curation Team. If you have thoughts or feedback, please share them in a comment here! If this PR has already been closed, you can start a new community contribution for this advisory


distutils was depreacted in Python 3.10

Ref migration advice : https://www.python.org/dev/peps/pep-0632/#migration-advice
https://github.com/WoLpH/portalocker/blob/b47d5f485f7c2e4d5af46307c6b3c4d2ea50e5f6/setup.py#L7
https://github.com/WoLpH/portalocker/blob/638f67af428413ef6b2a45633b6e4f034b11f272/portalocker_tests/test_combined.py#L5

You're right, this needs an update.
I don't have time to create a new release right now, but for my own reference. It looks like the LooseVersion can be replaced with: packaging.version.parse