Propagate configurations from master to clients automatically

High level design:
Each FileSystemContext has a background heartbeat thread checking whether cluster and path configurations have updated.
If they are updated, and there are no ongoing FileSystem operations (either unary or streaming RPC calls), the new configurations will be fetched from master and FileSystemContext will be reinitialized.
If there are active FileSystem operations, then the heartbeat will be blocked from updating configuration until there is no active operation or timeout.
Major changes:

ConfigHashSync is the heartbeat thread checking configuration hashes and trigger reinitialization of FileSystemContext;
FileSystemContextReinitializer contains the above heartbeat and locking logic to coordinate reinitializing the context;
FileSystemContext#reinit reinitializes the context;
MasterClientContext in FileSystemContext used to duplicate references to members in ClientContext, this causes inconsistencies when the context is reinitialized, this PR makes all master clients to respect a single source of truth of MasterClientContext.
BaseFileSystem#rpc takes a functional interface to abstract out all RPC calls;
Optimization in grpc service: one is to be able to request cluster level only or path level only configurations, another is to cache serialized protobuf of the configurations in MetaMasterConfigurationServiceHandler to mitigate serialization cost when clients try to load the updated configurations concurrently.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3296/
Test FAILed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3339/
Test FAILed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3340/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3342/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3343/
Test FAILed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3341/
Test FAILed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3344/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3345/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3346/
Test FAILed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3352/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3359/
Test FAILed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3362/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3364/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3366/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3384/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3389/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3390/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3391/
Test FAILed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3392/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3393/
Test PASSed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3394/
Test FAILed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3395/
Test FAILed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3397/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3399/
Test FAILed.

jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3402/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3403/
Test PASSed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3404/
Test FAILed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3409/
Test PASSed.

jenkins, test this please

jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3410/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3411/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3413/
Test PASSed.

jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3416/
Test PASSed.

jenkins, test this please

jenkins, test this please

@madanadit PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3477/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3479/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3478/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3480/
Test FAILed.

@madanadit PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3481/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3482/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3484/
Test PASSed.

@ggezer PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3587/
Test PASSed.

alluxio-bot, merge this please


Fixing issues with some unity components

on ParseEndPointReference in AbstractMemberBinding, if the components namespace starts with "UnityEngine", it removes the namespace and just uses the component name
I've noticed using GetComponent("") will not work for all unity components if you include the namespace (in my case, RectTransform), however using GetComponent("") and removing all unity namespaces still works on all unity components, so if a component starts with "UnityEngine." I remove all namespaces. Everything still seems to work fine, and you can now bind to RectTransform properties

Interesting. I just tested that out in the latest version of Unity (2017.2.0f3) and you're right, it doesn't seem to be able to get certain components when specified with the namespace.
I'll merge this but since it's a rather unintuitive thing to be doing unless you know the (undocumented) Unity behaviour, I think it's probably worth adding a comment first explaining why we're removing the namespace in certain conditions.
I think it would probably also be better if you didn't reuse the same lastPeriodIndex variable to mean two different things in different places, because that could be confusing if later on we need to expand that function further. Maybe just put the call to LastIndexOf() inline inside the call to .Substring().
Thanks again!

Alright, made those changes :)

Thanks! I just merged it.


Use seconds as time unit and add test

Hi,
the units for the time parameter temporal_threshold are documented as seconds in main.py which
is not consistent with the implementation.
I have changed the code to use seconds and added a minimal unit test which fails with the original parameterization.

Hi, Thank you for your contribution.


Feeding 2D tracks for reconstruction

Hi Paul,
Thanks for your great work. It is really helpful.
I was wondering is it possible to use 2D tracks (probably computed by PixSfM or other algorithms) for reconstruction? I am trying to do this by modifying extract_features.main to features_from_track and match_features.match_from_paths to match_from_paths_by_track.
Generally tracks is a tensor with the shape of [B, N, 2], where B is the number of image frames, N is the number of tracks, 2 is xy. I tried to feed tracks whose average sub-pixel accuracy is around 50%, i.e., half of the points have < 1 pixel distance to the ground truth. However, the reconstruction result is not very satisfactory. On MegaDepth, its AUC10 is around 45% while SP&SG is around 65%.
Do you think the less-than-ideal results stem from the inaccuracy of the tracks, or should I be looking into a more refined method of integrating them? Are there specific hyperparameters should be changed? For example, my matching scores come from cosine similarity which is usually >0.7, while I noticed the original matching scores may be quite small. Or generally, should we try to have more keypoints and matches, or filter the existing one as much as we can?
Thank you for your guidance and insights.
Best Regards
def features_from_track(conf: Dict,
         image_dir: Path,
         export_dir: Optional[Path] = None,
         as_half: bool = True,
         image_list: Optional[Union[Path, List[str]]] = None,
         feature_path: Optional[Path] = None,
         overwrite: bool = False,
         tracks = None,
         confidence = None,
         des = None,) -> Path:
    logger.info('Extracting local features with configuration:'
                f'\n{pprint.pformat(conf)}')

    dataset = ImageDataset(image_dir, conf['preprocessing'], image_list)
    if feature_path is None:
        feature_path = Path(export_dir, conf['output']+'.h5')
    feature_path.parent.mkdir(exist_ok=True, parents=True)
    skip_names = set(list_h5_names(feature_path)
                     if feature_path.exists() and not overwrite else ())
    dataset.names = [n for n in dataset.names if n not in skip_names]
    if len(dataset.names) == 0:
        logger.info('Skipping the extraction.')
        return feature_path

    device = 'cuda' if torch.cuda.is_available() else 'cpu'

    loader = torch.utils.data.DataLoader(
        dataset, num_workers=1, shuffle=False, pin_memory=True)

    for idx, data in enumerate(tqdm(loader)):
        name = dataset.names[idx]
        # I rename all the images by order
        # and save them as 0000.png, 0001.png ...
        # so that I can easily get their original index here
        our_idx = int(os.path.basename(name)[:4])

        pred = {}
        # to colmap
        pred['keypoints'] = tracks[0,our_idx].cpu().numpy() + 0.5
        confidence[confidence<0.5] = 0

        pred['scores'] = confidence[0, our_idx,].cpu().numpy()
        pred['descriptors'] = des[our_idx].permute(1,0).cpu().numpy()
        uncertainty = confidence[0, our_idx,].mean().cpu().numpy() / 2
        pred['image_size'] = original_size = data['original_size'][0].numpy()

        if as_half:
            for k in pred:
                dt = pred[k].dtype
                if (dt == np.float32) and (dt != np.float16):
                    pred[k] = pred[k].astype(np.float16)

        with h5py.File(str(feature_path), 'a', libver='latest') as fd:
            try:
                if name in fd:
                    del fd[name]
                grp = fd.create_group(name)
                for k, v in pred.items():
                    grp.create_dataset(k, data=v)
                if 'keypoints' in pred:
                    grp['keypoints'].attrs['uncertainty'] = uncertainty
            except OSError as error:
                if 'No space left on device' in error.args[0]:
                    logger.error(
                        'Out of disk space: storing features on disk can take '
                        'significant space, did you enable the as_half flag?')
                    del grp, fd[name]
                raise error

        del pred

    logger.info('Finished exporting features.')
    return feature_path

@torch.no_grad()
def match_from_paths_by_track(conf: Dict,
                     pairs_path: Path,
                     match_path: Path,
                     feature_path_q: Path,
                     feature_path_ref: Path,
                     overwrite: bool = False,
                     tracks = None,
                     cos_simi = None,
                     des = None,) -> Path:
    logger.info('Matching local features with configuration:'
                f'\n{pprint.pformat(conf)}')

    if not feature_path_q.exists():
        raise FileNotFoundError(f'Query feature file {feature_path_q}.')
    if not feature_path_ref.exists():
        raise FileNotFoundError(f'Reference feature file {feature_path_ref}.')
    match_path.parent.mkdir(exist_ok=True, parents=True)

    assert pairs_path.exists(), pairs_path
    pairs = parse_retrieval(pairs_path)
    pairs = [(q, r) for q, rs in pairs.items() for r in rs]
    pairs = find_unique_new_pairs(pairs, None if overwrite else match_path)
    if len(pairs) == 0:
        logger.info('Skipping the matching.')
        return

    device = 'cuda' if torch.cuda.is_available() else 'cpu'

    dataset = FeaturePairsDataset(pairs, feature_path_q, feature_path_ref)
    loader = torch.utils.data.DataLoader(
        dataset, num_workers=5, batch_size=1, shuffle=False, pin_memory=True)
    writer_queue = WorkQueue(partial(writer_fn, match_path=match_path), 5)

    for idx, data in enumerate(tqdm(loader, smoothing=.1)):
        # Pick the original idx of each pair
        left_idx = int(os.path.basename(pairs[idx][0])[:4])
        right_idx = int(os.path.basename(pairs[idx][1])[:4])

        pair = names_to_pair(*pairs[idx])
        pred = {}

        # Tracks (B, N, 2) have already be sorted
        # so their matches should be 0 - 0, 1 - 1, n - n
        pred['matches0'] = torch.arange(len(des)).to(device)
        pred['matches1'] = torch.arange(len(des)).to(device)

        # cos_simi (N, B, B) to (N, 1, 1)
        # cos_simi is cosine similarity, normalized to 0-1
        cross_conf = cos_simi[:,left_idx, right_idx]
        cross_conf = cross_conf * cross_conf
        invalid_mask = cross_conf < 0.75

        pred['matches0'][invalid_mask] = -1
        pred['matches1'][invalid_mask] = -1

        pred['matches0'] = pred['matches0'][None]
        pred['matches1'] = pred['matches1'][None]

        # the matching score is similarity
        cross_conf[invalid_mask] = 0
        pred['matching_scores0'] = cross_conf[None].to(device)
        pred['matching_scores1'] = cross_conf[None].to(device)

        writer_queue.put((pair, pred))
    writer_queue.join()
    logger.info('Finished exporting matches.')

Hi Paul,
Thanks for the quick response. Yes, I tried the track number of 256, 1024, and 4096, where 4096 shows the best result of AUC@10=45.
The tracks are distributed over the entire image. Here is an example from IMC:

And here is an sample to show the quality of tracks. The top is pred and the bottom is gt.

Its quality looks good to me and hence I was a bit confused about the COLMAP performance. I guess I need to double check some details :D

By the way, when the keypoints are defined in the original resolution, do we still need to specially deal with the keypoints scales? I noticed there is a step for keypoint scale in extract_features.main. I think this is because SP will resize the image to 1024, and we need to rescale the SP keypoints back to the original resolution, e.g., 512x512.  If tracks are already set in the 512x512 pixel coordinate, is scale adjustment still needed?

By the way, when the keypoints are defined in the original resolution, do we still need to specially deal with the keypoints scales?

No scaling is needed when the points are at the original image resolution.

The top is pred and the bottom is gt.

What happens when you feed the ground truth tracks? are you sure that this ground truth is sub-pixel accurate? In phototourism datasets the baseline between images is rather small so having well-localized points is critical.
Since these tracks are so good, you could also disable the geometric verification with hloc.reconstruction.main(..., skip_geometric_verification=True).
If this doesn't help, the next steps is to check how many of these ground truth tracks survive the reconstruction process by inspecting the number of points reconstructed by each image in the final 3D model. Looking at statistics printed by COLMAP during incremental mapping would also be useful.


Listen to height changes.

Listen to height changes on each toast to avoid bugs like this one - https://github.com/emilkowalski/sonner/issues/66#issuecomment-1548194697.

Fixed in #59


add ability to use bytes type in avro

Codecov Report

Merging #55 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #55   +/-   ##
=======================================
  Coverage   92.39%   92.39%
=======================================
  Files         118      118
  Lines        1761     1761
  Branches      100      100
=======================================
  Hits         1627     1627
  Misses        134      134

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 065ef2d...f2fe371. Read the comment docs.

This isn't going to work.  For now, we won't support bytes type.


plasma5Packages.plasmaMobileGear: 22.11 -> 23.01.0

Description of changes

https://plasma-mobile.org/2023/01/30/january-blog-post/

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

cc @matthiasbeyer

Important notes from their announcements:

Housekeeping
As announced in the last blog post, we had decided to migrate the releases of Plasma Mobile applications to KDE Gear, starting with KDE Gear 23.04 (in April).
Because of this, we will now decouple the blog post format from any sort of software release schedule, and try to get one out at least on a bi-monthly basis!
The blog post you are currently reading still coincides though with the release of Plasma Mobile Gear 23.01, which is the last Mobile Gear release.

So uuuuuuh... does this mean we should drop that package set since it'll be in Plasma Gear? cc @NixOS/qt-kde

Testing results:
Works

alligator
angelfish
audiotube
calindori
kalk
kasts
kclock
keysmith
koko
krecorder
ktrip [launches, can't trivially test]
neochat [launches, logging-in is a pain]
plasma-phonebook
plasma-dialer [launches, don't have a phone line setup]
plasmatube
spacebar [launches, no sms setup]
tokodon

Issues
kweather
Needed to delete my config to get something in the window... Once done, I can't add a city.
qrc:/qml/ForecastContainerPage.qml:37: TypeError: Value is null and could not be converted to an object

Also it seems there's an implicit new dependency (maybe related)
qrc:/qml/DynamicForecastPage.qml:585:17: Type SunriseCard unavailable
qrc:/qml/SunriseCard.qml:15:1: module "org.kde.kholidays" is not installed

I can't add a city

I added kholidays and it worked for me. Can you try again?

Under which "environment" (DE, env vars, everything) are you running kweather?

i3, so Xorg

Just verified kweather with kholidays fixes the issue with it not working. Though I thought it might have had another underlying issue, it ended-up being the same cause.


Why did I get this error, refresh the page and freeze with the same? Is it a bug?

Are you sure that you haven't changed something in the Collection class file in your vendor folder? Because the method jsonSerialize is clearly there -> https://github.com/laravel/framework/blob/5.6/src/Illuminate/Support/Collection.php#L1716 so I don't see any other possibility as to how you could get that error.

it fixed when I reloaded my server


Add support for mixedbread-ai/mxbai-embed-large-v1

Add support for https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1.
https://www.mixedbread.ai/blog/mxbai-embed-large-v1

Hi Prabir. Thanks for raising the issue.
I am deep into work this week. If anyone is interested in a contribution, a PR could look like
https://github.com/Anush008/fastembed-rs/pull/36
Plus, any optional modification this model might need.


Easily allow customized losses

Facilitate the implementation of models that need custom losses --right now the loss is mainly defined in dataset config files.

I believe we define the type of loss, instead, we can move it to the module structure similar for "encoder" and "readout"

Addressed in #112


Dealing with business hours that are or cross midnight

There seems to be no way to define hours that cross over midnight and still return a precise time. I could set the end time to 23:59:59, but this workaround seems a little messy.
One potential solution would be to assume that if the start time is greater than the end time that you need to add a day, but I haven't fully examined how you're implementing your calculations to determine if that will work.

To clarify, if your normal business hours are 2200 to 0600, then there is not any way I see to configure that properly.

Similarly, there seems to be no way to address operations that are 24x7 or 24x5.

I ran into the same problem and worked around it using the following logic:
`zero = time(0)
one = time(23, 59, 59)
open_time = (datetime.combine(date.today(), zero) + _open).time()
close_time = (datetime.combine(date.today(), zero) + _close).time()
open_days = [d - 1 for d in self.weekdays if not d in closed_days]
if _close < _open:
businesshrs = Rules([
WorkDayRule(zero, close_time, working_days=open_days),
WorkDayRule(open_time, one, working_days=open_days)
])
else:
businesshrs = WorkDayRule(open_time, close_time, working_days=open_days)`

Thanks Matthew and Ruben. With a (very long) delay (sorry) your suggested solution is implemented in 238b272 , and released in businesstimedelta version 1.0.1.


Warnings in github actions

We have a lot of warnings about deprecated github actions on CI: https://github.com/networkservicemesh/integration-k8s-kind/actions/runs/7843310592#:~:text=checkgomod / checkgomod

Seems like done.


(Bug and Suggestion?) - Breaks controls

Might be specific to Oculus VR, but when playing, all grab controls are broken, x and b symbols are added on repeatedly to the controls list without vanishing (the icons just keep adding on and getting smaller), and the recorder or writer thing doesn't seem to work at all? Not sure about the chat feature, I don't know if anyone was on or responding when I typed but you should be able to see the chat at all times, which I couldn't because it's small and in my peripheral at all times. I also couldn't pause.
It would be nice if you could rebind every control to different buttons or sets of buttons like instead of bumper+B for chat, bumper+trigger+Y, and save configs, though I know that could be hard or could take longer to implement
i would appreciate it if this was fixed because online is unplayable without it
List:
X and B icons keep showing up

recorder doesn't work
can't grab anything
https://user-images.githubusercontent.com/68121578/175988382-e788ca50-eb80-4264-a025-306b69781437.mp4
can't pause

bump - had this issue a lot on oculus q2


boards: stm32f769i_disco: Add SDMMC support

Define sdmmc2 for stm32f7 series.
Define board specific config for sdmmc2 module.

Tested using zephyr/samples/subsys/fs/fat_fs/ and MicroSD Card formatted as FAT32.
Signed-off-by: Mohamed ElShahawi<EMAIL_ADDRESS>

@ABOSTM @FRASTM Can you please review and approve ?


Fix issue #6699, 'Stay signed in for two weeks' not working

Fixes issue #6699.
The ccmAuthUserhash cookie was only evaluated when a session existed, which usually does not since the GDPR changes, so now it gets evaluated if there is no session or no uID set in the session.
SessionValidator needed to be changed because with the above changes we have a started session but haven't set the session cookie in $app['cookies'].

I've changed it like you suggested. Seems to work fine.
Still, $session = $this->app['session'] did not seem to start the session, at least $session->isStarted() is false and no session cookie is set. Only accessing it via $session->has('uID') or similar did it for me.
There is a side effect: If the ccmAuthUserHash cookie is expired or invalid, a session is still started with the CONCRETE5 cookie. Would that be a GDPR or otherwise an issue? Or maybe not because there had to be a valid session some time before?

@Xianghar This seems like a reasonable tradeoff to me, since it's obvious you had an active session at some time.


Reorganize tests to follow foundry best practices

Naming and also making sure we target the appropriate test cases

https://github.com/ScopeLift/optimizoors-perpetual-protocol/pull/13#discussion_r1194043289 make sure to do inverse fuzz tests

@alexkeating looks like you opened a PR for this, so I'm going to reassign this issue to you


/gs not working?

It is highly possible that this is user error. But I am unable to use "/gs" command to bring up the options to import macro's and what not..
Other commands like
"/gs help" works
"/gs showspec" works
Any help would be greatly appreciated

Hi NordicReign
In your add ons check that GSE_GUi is enables.
TimothyLuke

On 11 Aug 2018, at 5:21 am, NordicReign<EMAIL_ADDRESS>wrote:
It is highly possible that this is user error. But I am unable to use "/gs" command to bring up the options to import macro's and what not..
Other commands like
"/gs help" works
"/gs showspec" works
Any help would be greatly appreciated
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


NTR: IAO:"Preferred root term" (mildly urgent)

There is a new long-awaited OLS feature in the oven which allows us to configure the root terms of the displayed ontology. This will reduce confusion of which hierarchies actually "belong" to an ontology, and enable the proper display of ontologies that are not built on a complete isa-backbone (i.e. where some classes are connected only via part of and so on).
The development branch currently (about to be merged) uses some homemade annotation property in the OLS namespace (something like https://www.ebi.ac.uk/ols/hasPreferredRootTerm); Terms that are annotated with this property will be displayed as roots. Since I expect this property to be widely deployed across OBO ontologies for display purposes, do you think IAO is a more appropriate home? Caveat: We should decide fairly quickly, to bot block the release of the feature. Thanks!
@cmungall @jamesaoverton @dosumis @zhengj2007 @simonjupp

Hurrah!
I actually prefer human readable IRIs for things like this, but currently we are using numerics, just grab yourself a range in the idspace file and add this. Even if it's not in an IAO release we can still agree on the URI.
Note preferred root is not synonymous with 'belonging', so it would be good to have clear labels and definitions (I guess the semantics free IRIs are better here...)

Once we agree this is roughly ok, even if it is not merged, we will start using the IRI (to unblock OLS Dev team).
https://github.com/information-artifact-ontology/IAO/pull/222

I am wondering whether we should add the new annotation property in the ontology-metadata.owl rather than iao.owl.
https://github.com/information-artifact-ontology/ontology-metadata

Oh I didn’t know this file existed! Agreed!

@matentzn Could you please add the term in this file:
https://github.com/information-artifact-ontology/ontology-metadata/blob/master/src/ontology/ontology-metadata-edit.owl
I can merge the change after you finish and may make the release in December.

https://github.com/information-artifact-ontology/ontology-metadata/pull/45
Thanks @zhengj2007

Request annotation, has ontology root term, has been added in with ID: IAO_0000700


To markdown options fix

Issue related : https://github.com/IonicaBizau/medium-editor-markdown/issues/26

Thanks a lot, @annayafi! Overall this looks amazing! :sparkles:
There are few minor things to be fixed–I commented inline. :four_leaf_clover:


Renderer aemgpt/components/page is missing in the code repository.

I have followed all the steps, after installing the package, when I try to hit http://localhost:4502/content/aemgpt/us/en.html
I get the following error message. I noticed that /apps/aemgpt has only clientlibs and osgiconfig, where are the components ?
where is the rendered for "aemgpt/components/page" defined ?
02.07.2023 22:14:31.624 ERROR [[0:0:0:0:0:0:0:1] [1688316271613] GET /content/aemgpt/us/en.html HTTP/1.1] org.apache.sling.servlets.get.impl.DefaultGetServlet No renderer for extension html, cannot render resource JcrNodeResource, type=aemgpt/components/page, superType=null, path=/content/aemgpt/us/en/jcr:content

Hi @shivakumar660
Should work now. As the project doesn't need the archetype components I removed them. Will add some additional examples in future!
Let me know if that works for you.

Jake

Hi Jake,
Thank you for the quick response.
I see that in the filter.xml of ui.apps, you have added the path till clientlibs "//apps/aemgpt/clientlibs" , so its not picking the components as part of deployment. ct?  I would like to contribute for bug fixes or new features development.
I have around 10 years of experience as a AEM Developer , here is my linkedIn - https://www.linkedin.com/in/shivakumar-chandrappa-3221642a/

Hi Shivakumar,
Feel free to make a PR! Happy to have any contributions.
The integration itself does not need any components added, so unless for examples there really isn't a need for any to be included in this package.
The author clientlib attaches to the components on this line:
https://github.com/jaketracey/opax/blob/7a7c50d5fc78b8c957b763272610979bda0ab80b/ui.frontend/src/main/webpack/site/main.js#L22C27-L22C40
Ultimately we want to modify this so that it uses the OSGI configuration so that it's configurable which component the integration attaches to (not just wcm/text/v2 as it currently is.

I am getting this message when I try to raise a PR
"Pull request creation failed. Validation failed: must be a collaborator"

Fixed - should work now!

https://github.com/jaketracey/opax/pull/7
I have created a PR.

PR is merged. Hence closing this issue.


[3118] Ignore ungeocoded sites in distance search

Context

https://trello.com/c/fYgK7vnQ/3118-sort-by-distance-only-uses-sites-that-have-been-geocoded
We are not excluding un-geocoded sites from distance sorting
These sites are then deemed be at the origin of the search query as a result

Changes proposed in this pull request

When sorting by distance we exclude any site that has not been geocoded
These are sites that do not have latitude and longitude present

Guidance to review

visit http://localhost:3002/results?fulltime=False&hasvacancies=True&l=1&lat=53.9588365&lng=-1.0794777&loc=York+YO1%2C+UK&lq=York+YO1%2C+UK&parttime=False&qualifications=QtsOnly%2CPgdePgceWithQts%2COther&rad=20&senCourses=False&sortby=2&subjects=31%2C32
Results should be ordered correctly

Checklist

[x] Make sure all information from the Trello card is in here
[x] Attach to Trello card
[x] Rebased master
[x] Cleaned commit history
[x] Tested by running locally

Use the github attach button in trello to get a proper two-way link between the card and PR.

Good work, test + fix. Just some code structure comments from me.

 Sort by distance only uses sites that have been geocoded


Pickling binded classes

Hey there,
I want to read data in a multiprocess setting. However, for many of the binded classes, there are no pickling methods. For example, for AriaDigitalTwinDataProvider. If I use it as is (assign it as a property to class which exists in a separate process), I get:
TypeError: cannot pickle '_adt_pybinds.AriaDigitalTwinDataProvider' object.
I tried patching it with:
class AriaDigitalTwinDataProviderWrapper(AriaDigitalTwinDataProvider):

    def __getstate__(self):
        return deepcopy(self.__dict__)

    def __setstate__(self, state):
        self.__dict__.update(state)

class AriaDigitalTwinDataPathsProviderWrapper(AriaDigitalTwinDataPathsProvider):

    def __getstate__(self):
        return deepcopy(self.__dict__)

    def __setstate__(self, state):
        self.__dict__.update(state)

however, it doesn't work. To test it I run gt_provider.get_start_time_ns(), then pickle, unpickle, and then run gt_provider.get_start_time_ns() and it gives me a wrong result. Moreover, accessing data from the unpickled object crashes without any messages (I assume seg fault). I guess that some of the dependencies of these classes also don't know how to pickle themselves?
Is there a nice way to overcome this? Potentially, I could preprocess the data offline in a format I need and re-use it in a multiprocess setting but I want to avoid it and use the tools out-of-the-box.

Hi @VladimirYugay, thanks for your question. We currently have no plans to support pickling for our classes in projectaria_tools, or in ADT specifically. If you would like, we can document this as a feature request to consider in future releases. Alternatively, you can try to implement this feature yourself and even contribute the feature back to the repo via PR. Here is the documentation to help you get started: https://pybind11.readthedocs.io/en/stable/advanced/classes.html#pickling-support

Hi @VladimirYugay, I'm going to close this task but please re-open if you have any further questions or comments. Thanks!


update version python

Description
Update python versions

I have just updated the yml-files, I have just assumed that if the github actions went through then it would work correctly, but that's maybe a wrong assumption @heidi-holm-4ss

I suggest you create an environment with python 3.11 and verifies that the tests work and that you are able to authenticate and download data

@heidi-holm-4ss
I have now run the application with python version 3.11.0. All the tests works.


Differentiating between multiple providers

Hi! First off, this is a fantastic library - thank you for all your efforts. If I define multiple providers for the same type and then pass both providers to the Injector, the "last" provider takes precedence (see below). While I am pretty sure this is intentional - and aligns nicely with how python handles similarly named functions - I was wondering if there was a way to a) differentiate between the providers or b) raise an exception if multiple providers have been defined for the same type. I would be happy to contribute a PR if you think there are any worthwhile features to be added. Thanks again!
from dataclasses import dataclass
from injector import Module, singleton, provider, Injector

@dataclass
class Output:
    value: str

class Provider01(Module):
    @singleton
    @provider
    def provide_output(self) -> Output:
        return Output("Using Provider01")

class Provider02(Module):
    @singleton
    @provider
    def provide_output(self) -> Output:
        return Output("Using Provider02")

injector = Injector([Provider01(), Provider02()])
output = injector.get(Output)
print(output.value)  # Using Provider02

What would you want to achieve in option a? I'm not sure it matches your use case, but I have previously used parent/child injectors to override providers.

Ah, I think I choose my words poorly in this case - my apologies. @davidparsson, your suggestion about using parent/child injectors is brilliant and I will absolutely keep that in mind when I have to, knowingly, override a provider. However, what I meant to ask in (a) was more along the lines of 'is there a way to detect that multiple providers have been bound to the same type'? I will edit the original question.

I used the "last processed module overrides whatever previous modules defined if there is an overlap" behavior in tests in the past.
That said, it seems like it'd be a good idea to at least provide an option to disallow that (by raising an exception) – using a switch in the Injector constructor, possibly?


Default SKIP filter is wrong

Steps to reproduce

New installation 1.7
Check Skip Filter
TF is has double parentheses by default

Expected behavior
$stricmp(%genre%,instrumental)$stricmp(%genre%,classical)
Versions

OpenLyrics version: 1.7

Meh. I saw this locally but I thought it was just the value I had saved, and didn't realise it was the default. I'll try get a bugfix release out soon (probably like next weekend or something) that address this and anything else major that comes up in the interim.
Thanks for reporting it!


Locate errors within macro definition.

I get the following with rust-analyzer on:

Whereas cargo check yields:
error: expected identifier, found keyword `fn`
 --> src/main.rs:3:16
  |
3 |         struct fn async {}
  |                ^^ expected identifier, found keyword
...
7 | test!();
  | -------- in this macro invocation
  |
  = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

Put it simply, the error is displayed at the macro invocation, but not within the macro definition.
I understand that it must be difficult and maybe sometimes ambiguous to locate errors within macro definitions. But since rustc seems able to locate the error at 3:16, could rust-analyzer not do the same?
It does not yet, which makes it difficult editing macros with large bodies, because no errors pop up within them like they usually do in regular code.

Should be fixed around here: https://github.com/rust-analyzer/rust-analyzer/blob/00c81f8006926b5a326d4dd11f61ea5ae09aadbd/crates/rust-analyzer/src/diagnostics/to_proto.rs#L43-L63


MY NAMI /new or seed phrase

I have gone to my NAMI wallet in chrome extensions, and it says welcome New  Wallet,,  Import ,when I click on Import and click accept terms it does not let me click on continue so I can enter my existing seed phrase, it only allows me to choose new seed phrases,
I do not want to do this as It will not bring my tokens/staked into a new seed phrase wallet, I don't know why my NAMI wallet has gone to a new wallet or import in the first place , but can you advise on this situation and how I can resolve it thanks

@damien100 Talk to Cardano Support on the Live support page so that your issue can be looked into.

Have you able to resolved your issues?

Have you able to resolved your issues?

Hello Mai87, no I have not are you able to advise,

@damien100 Did you talk to the live support yet about your issues?

I am concerned  I just noticed this on your website address,

It is caused either by VPN or IP on your device it's nothing you shouldn't worry about it.

I am concerned I just noticed this on your website address,

The best hacker who helped me with that I will advise message him on telegram
https://t.me/Blackhathacker110


Torch nondeterministic segfault

Found another segfault with the ThreadedEnginePerDevice Executor. It does not happen on the NaiveEngine. It also appears to only happen when working with torch criterion. It appears to only occur when adding two criterion (MSE cost plus softmax for an autoencoder classifier model) after around 400 iterations of successful running.
Could there be a race in the order in which these two criterion are run  / issues if they are run at the same time?
My classification head looks something like:
    flatten = some_big_model()
    fc1 = mx.symbol.FullyConnected(data=flatten, num_hidden=num_classes, name='fc1')

    label = mx.sym.Variable(name="softmax_label")

    logsoftmax = mx.symbol.TorchModule(data_0=fc1, lua_string='nn.LogSoftMax()', num_data=1, num_params=0, num_outputs=1, name='logsoftmax')
    label = mx.symbol.Variable('softmax_label') + 1
    softmax = mx.symbol.TorchCriterion(data=logsoftmax, label=label, lua_string='nn.ClassNLLCriterion()', name='softmax')
    recon = some_big_reconstruction_deconv(in5b)

    mse = mx.symbol.TorchCriterion(data=recon, label=data/150., lua_string='nn.MSECriterion()', name="mse", label_shape=(3, 64, 64))

    cost = softmax + mse*10

    D = lambda x: mx.symbol.BlockGrad(x)
    last = D(logsoftmax)

    final_symbol = mx.symbol.Group([cost])

Backtrace from GDB
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffe85ff3700 (LWP 29323)]
musable (mem=0x7ffe60891870) at malloc.c:4570
4570	malloc.c: No such file or directory.
(gdb) backtrace
Python Exception <class 'ImportError'> This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.:
#0  musable (mem=0x7ffe60891870) at malloc.c:4570
#1  __malloc_usable_size (m=0x7ffe60891870) at malloc.c:4581
#2  0x00007fffc138e029 in THFree ()
   from /home/luke/torch/install/lib/libTH.so
#3  0x00007fffbf5b9f08 in THCudaTensor_free ()
   from /home/luke/torch/install/lib/libTHC.so
#4  0x00007fffa9e15ca0 in torch_CudaTensor_free ()
   from /home/luke/torch/install/lib/lua/5.1/libcutorch.so
#5  0x00007fffc1a381fa in lj_BC_FUNCC ()
   from /home/luke/torch/install/lib/libluajit.so
#6  0x00007fffc19fa96a in gc_onestep ()
   from /home/luke/torch/install/lib/libluajit.so
#7  0x00007fffc19faabc in lj_gc_step ()
   from /home/luke/torch/install/lib/libluajit.so
#8  0x00007fffc1a21663 in lua_newuserdata ()
   from /home/luke/torch/install/lib/libluajit.so
#9  0x00007fffc17b2f7c in luaT_pushudata ()
   from /home/luke/torch/install/lib/libluaT.so
#10 0x00007fffccedddd6 in mxnet::TorchState::Deserialize(THCharStorage*) ()
   from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#11 0x00007fffcd0e7136 in mxnet::op::TorchCriterionOp<mshadow::gpu>::Forward (this=0x385207b0, ctx=..., in_data=..., req=...,
    out_data=..., aux_args=...)
    at plugin/torch/./torch_criterion-inl.h:79
#12 0x00007fffcce9da22 in mxnet::GraphExecutor::GetOpExecEntry(unsigned int)::{lambda(mxnet::RunContext, mxnet::engine::CallbackOnComplete)#1}::operator()(mxnet::RunContext, mxnet::engine::CallbackOnComplete) const ()
   from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#13 0x00007fffcccbca89 in mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext, mxnet::engine::OprBlock*) ()
   from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#14 0x00007fffcccbcf20 in std::_Function_handler<void (), mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*, bool)::{lambda()#3}::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
   from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#15 0x00007fffbf236a40 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#16 0x00007ffff7bc4182 in start_thread (arg=0x7ffe85ff3700)
    at pthread_create.c:312
#17 0x00007ffff78f147d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Possibly related to: #1242

@mavenlin Maybe we should consider make separate lua_state per operator?

I will give it a try.

Actually there might be another cause. It failed in malloc and I think
malloc is non reentrant. So that might be the problem.
On Feb 3, 2016 6:47 PM, "Luke"<EMAIL_ADDRESS>wrote:

I will give it a try.
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1413#issuecomment-179588356.

Also, why would a shared Lua_State matter if they are never used across threads? In both an operator specific Lua_State and thread specific Lua_State there should be no races.

You are probably right. So may be its non reentant malloc problem. Or
something else.
On Feb 3, 2016 7:09 PM, "Luke"<EMAIL_ADDRESS>wrote:

Also, why would a shared Lua_State matter if they are never used across
threads? In both an operator specific Lua_State and thread specific
Lua_State there should be no races.
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1413#issuecomment-179592596.

@piiswrong:
I have been trying to do a bit of investigating. My best guess as to what is going on is a race in when the lua garbage collector that is only getting triggered because an operator needs the output of two criterion.
I did a small test of putting a collectgarbage("collect") call into sharedserialize.lua save and load functions just before the return. This does create a reproducible segfault.
I am not sure why this is happening but is a problem.

is the lua garbage collect specific to lua states or a global background
thread?
On Feb 4, 2016 11:11 AM, "Luke"<EMAIL_ADDRESS>wrote:

@piiswrong https://github.com/piiswrong:
I have been trying to do a bit of investigating. My best guess as to what
is going on is a race in when the lua garbage collector that is only
getting triggered because an operator needs the output of two criterion.
I did a small test of putting a collectgarbage("collect") call into
sharedserialize.lua save and load functions just before the return. This
does create a reproducible segfault.
I am not sure why this is happening but is a problem.
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1413#issuecomment-180006280.

What about two torch module instead of torch criterion?
On Feb 4, 2016 12:09 PM, "eric xie"<EMAIL_ADDRESS>wrote:

is the lua garbage collect specific to lua states or a global background
thread?
On Feb 4, 2016 11:11 AM, "Luke"<EMAIL_ADDRESS>wrote:

@piiswrong https://github.com/piiswrong:
I have been trying to do a bit of investigating. My best guess as to what
is going on is a race in when the lua garbage collector that is only
getting triggered because an operator needs the output of two criterion.
I did a small test of putting a collectgarbage("collect") call into
sharedserialize.lua save and load functions just before the return. This
does create a reproducible segfault.
I am not sure why this is happening but is a problem.
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/1413#issuecomment-180006280.

I think the lua_state are completely separate and thus would have a separate garbage collector. (http://stackoverflow.com/questions/17653530/is-it-possible-to-share-a-table-between-multiple-lua-state-in-a-single-threaded)
The crash does appear to happen for both criterion and modules but its hard to 100% confirm. My test case is as follows:
import mxnet as mx

def crash_combo_criterion():
    data = mx.sym.Variable(name="data")
    label = mx.sym.Variable(name="softmax_label")

    fc1 = mx.symbol.FullyConnected(data=data, num_hidden=10, name='fc1')
    logsoftmax = mx.symbol.TorchModule(data_0=fc1, lua_string='nn.LogSoftMax()', num_data=1, num_params=0, num_outputs=1, name='logsoftmax')
    # Torch's label starts from 1
    softmax = mx.symbol.TorchCriterion(data=logsoftmax, label=label+1, lua_string='nn.ClassNLLCriterion()', name='softmax')
    mse = mx.symbol.TorchCriterion(data=data, label=data+1, lua_string='nn.MSECriterion()', name="mse", label_shape=(10,))
    cost = softmax + mse*10

    executor = cost.simple_bind(data=(12, 10), softmax_label=(12,), ctx=mx.gpu(3))

    for i in range(1000):
        executor.forward()
        executor.backward()
        executor.outputs[0].asnumpy()

def crash_modules():
    data = mx.sym.Variable(name="data")

    fc1 = mx.symbol.FullyConnected(data=data, num_hidden=10, name='fc1')
    logsoftmax = mx.symbol.TorchModule(data_0=fc1, lua_string='nn.LogSoftMax()', num_data=1, num_params=0, num_outputs=1, name='logsoftmax')
    logsoftmax2 = mx.symbol.TorchModule(data_0=fc1, lua_string='nn.LogSoftMax()', num_data=1, num_params=0, num_outputs=1, name='logsoftmax')

    cost = logsoftmax + logsoftmax2

    executor = cost.simple_bind(data=(12, 10), ctx=mx.gpu(3))
    out = mx.nd.zeros((12, 10), ctx=mx.gpu(3))

    for i in range(1000):
        executor.forward()
        executor.backward([out])
        executor.outputs[0].asnumpy()

crash_combo_criterion()
#crash_modules()

I was unable to get these to crash however without a modification to the lua threads library.
The modification is to add a collectgarbage("collect") call just before the return in serialize.load and serialize.save. It is my understanding is telling lua to collect garbage should never release used instances and should never cause a crash.
The error message is slightly different than before

*** Error in `/usr/bin/python': realloc(): invalid pointer: 0x000000002d4be2a0 ***

Program received signal SIGABRT, Aborted.
0x00007ffff782dcc9 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56	../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace
Python Exception <class 'ImportError'> This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.:
#0  0x00007ffff782dcc9 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff78310d8 in __GI_abort () at abort.c:89
#2  0x00007ffff786a394 in __libc_message (do_abort=do_abort@entry=1,
    fmt=fmt@entry=0x7ffff7978b28 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff78750f7 in malloc_printerr (action=<optimized out>,
    str=0x7ffff7974d5d "realloc(): invalid pointer", ptr=<optimized out>)
    at malloc.c:4996
#4  0x00007ffff787a192 in __GI___libc_realloc (oldmem=0x2d4be2a0, bytes=16)
    at malloc.c:2986
#5  0x00007fffc070b080 in THRealloc () from /home/luke/torch/install/lib/libTH.so
#6  0x00007fffc07218d4 in THFloatTensor_rawResize.constprop.246 ()
   from /home/luke/torch/install/lib/libTH.so
#7  0x00007fffa946f9d4 in THNN_FloatLogSoftMax_updateOutput ()
   from /home/luke/torch/install/lib/lua/5.1/libTHNN.so
#8  0x00007fffc0db71b9 in lj_vm_ffi_call ()
   from /home/luke/torch/install/lib/libluajit.so
#9  0x00007fffc0d8a2ef in lj_ccall_func ()
   from /home/luke/torch/install/lib/libluajit.so
#10 0x00007fffc0d8c976 in lj_cf_ffi_meta___call ()
   from /home/luke/torch/install/lib/libluajit.so
#11 0x00007fffc0db51fa in lj_BC_FUNCC ()
   from /home/luke/torch/install/lib/libluajit.so
#12 0x00007fffc0da429d in lua_pcall ()
   from /home/luke/torch/install/lib/libluajit.so
#13 0x00007fffccf408e9 in mxnet::op::TorchModuleProp::InferShape(std::vector<mshadow::TShape, std::allocator<mshadow::TShape> >*, std::vector<mshadow::TShape, std::allocator<mshadow::TShape> >*, std::vector<mshadow::TShape, std::allocator<mshadow::TShape> >*) const () from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#14 0x00007fffccead136 in mxnet::StaticGraph::InferNodeShapes(std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<std::vector<mshadow::TShape, std::allocator<mshadow::TShape> >, std::allocator<std::vector<mshadow::TShape, std::allocator<mshadow::TShape> > > >*, std::vector<std::vector<mshadow::TShape, std::allocator<mshadow::TShape> >, std::allocator<std::vector<mshadow::TShape, std::allocator<mshadow::TShape> > > >*) const ()
   from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#15 0x00007fffcce8bc78 in mxnet::GraphExecutor::InitDataEntryInfo(std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&) () from /home/luke/Repos/dmlc_mxnet/python/mxnet/../../lib/libmxnet.so
#16 0x00007fffccea7568 in mxnet::GraphExecutor::Init(mxnet::Symbol, mxnet::Context const&, std::map<std::string, mxnet::Context, std::less<std::string>, std::allocator<std::pair<std::string const, mxnet::Context> > > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&) ()

Here is a temporary work around: Since crash happens in serialization/deserialzation, we can remove the need for them by making a lua_state per operator, instead of per thread.
The down side of this is every operator will have to load a copy of torch. But hopefully torch lib doesn't take up that much memory.

Just submitted a PR that does just this. I gave up trying to track down the bad references. I think its something to do with the output variable on the nn.Module but I am not sure.


Ban from Telegram when authorise/monitor multiple accounts with TDLib on Java

hey guys,
Based on this comment, to authorize multiple accounts with TDLib, we can create as many TDLib clients as I need and initialize them with different database folders.
Bellow plain client to authorize multiple accounts based on this example
import java.io.FileNotFoundException;
import java.io.IOError;
import java.io.IOException;
import java.util.Arrays;
import org.drinkless.tdlib.TdApi;
import org.drinkless.tdlib.TdApi.AuthorizationState;
import org.drinkless.tdlib.TdApi.AuthorizationStateClosed;
import org.drinkless.tdlib.TdApi.AuthorizationStateClosing;
import org.drinkless.tdlib.TdApi.AuthorizationStateLoggingOut;
import org.drinkless.tdlib.TdApi.AuthorizationStateReady;
import org.drinkless.tdlib.TdApi.AuthorizationStateWaitEncryptionKey;
import org.drinkless.tdlib.TdApi.AuthorizationStateWaitOtherDeviceConfirmation;
import org.drinkless.tdlib.TdApi.AuthorizationStateWaitPhoneNumber;
import org.drinkless.tdlib.TdApi.AuthorizationStateWaitTdlibParameters;
import org.drinkless.tdlib.TdApi.CheckDatabaseEncryptionKey;
import org.drinkless.tdlib.TdApi.Function;
import org.drinkless.tdlib.TdApi.Object;
import org.drinkless.tdlib.TdApi.RequestQrCodeAuthentication;
import org.drinkless.tdlib.TdApi.SetTdlibParameters;
import org.drinkless.tdlib.TdApi.TdlibParameters;
import org.drinkless.tdlib.TdApi.UpdateAuthorizationState;
import org.drinkless.tdlib.TdApi.UpdateNewMessage;
import org.springframework.util.ResourceUtils;

public final class Client implements Runnable {

    static {
        try {
            System.load(ResourceUtils.getFile("classpath:libs/mac/amd64/tdjni.dylib").getAbsolutePath());
            nativeClientExecute(new TdApi.SetLogVerbosityLevel(0));
            if (nativeClientExecute(new TdApi.SetLogStream(new TdApi.LogStreamFile("logs/tdlib.log", 1 << 27))) instanceof TdApi.Error) {
                throw new IOError(new IOException("Write access to the current directory is required"));
            }
        } catch (FileNotFoundException e) {
            throw new RuntimeException("Unable to execute static initialization");
        }
    }

    private static native long createNativeClient();

    private static native void nativeClientSend(long nativeClientId, long eventId, TdApi.Function function);

    private static native int nativeClientReceive(long nativeClientId, long[] eventIds, TdApi.Object[] events, double timeout);

    private static native TdApi.Object nativeClientExecute(TdApi.Function function);

    private static native void destroyNativeClient(long nativeClientId);

    private final long nativeClientId;
    private final long databaseId;

    public Client(long databaseId) {
        this.nativeClientId = createNativeClient();
        this.databaseId = databaseId;
    }

    public void poll() {
        Arrays.stream(receive()).forEach(response -> {
            if (response == null || response.getObject() == null) {
                return;
            }
            if (response.getObject() instanceof UpdateAuthorizationState) {
                UpdateAuthorizationState updateAuthorizationState = (UpdateAuthorizationState) response.getObject();
                handleAuthorizationState(updateAuthorizationState.authorizationState);
            } else if (response.getObject() instanceof UpdateNewMessage) {
                UpdateNewMessage updateNewMessage = (UpdateNewMessage) response.getObject();
                System.out.println(updateNewMessage.message.content.toString());
            }
        });
    }

    @Override
    public void run() {
        while (true) {
            poll();
        }
    }

    private void handleAuthorizationState(AuthorizationState authorizationState) {
        switch (authorizationState.getConstructor()) {
            case AuthorizationStateWaitTdlibParameters.CONSTRUCTOR: {
                TdlibParameters parameters = new TdlibParameters();
                parameters.databaseDirectory = "database/" + databaseId;
                parameters.useMessageDatabase = true;
                parameters.useSecretChats = true;
                parameters.apiId = 1503950;
                parameters.apiHash = "86a6e7681d15ae34089b2bf9b3014188";
                parameters.systemLanguageCode = "en";
                parameters.deviceModel = "MacOS";
                parameters.systemVersion = "10.15.6";
                parameters.applicationVersion = "1.0.1";
                parameters.enableStorageOptimizer = true;
                send(new Request(2L, new SetTdlibParameters(parameters)));
                break;
            }
            case AuthorizationStateWaitEncryptionKey.CONSTRUCTOR: {
                send(new Request(3L, new CheckDatabaseEncryptionKey()));
                break;
            }
            case AuthorizationStateWaitPhoneNumber.CONSTRUCTOR: {
                send(new Request(4L, new RequestQrCodeAuthentication()));
                break;
            }
            case AuthorizationStateWaitOtherDeviceConfirmation.CONSTRUCTOR: {
                String link = ((AuthorizationStateWaitOtherDeviceConfirmation) authorizationState).link;
                System.out.println("Please confirm this login link '" + link + "' on another device; id='" + databaseId + "' ");
                break;
            }

            case AuthorizationStateReady.CONSTRUCTOR: {
                System.out.println("Authorised; id='" + databaseId + "' ");
                break;
            }
            case AuthorizationStateLoggingOut.CONSTRUCTOR: {
                System.out.println("Logging out; id='" + databaseId + "' ");
                break;
            }
            case AuthorizationStateClosing.CONSTRUCTOR: {
                System.out.println("Closing; id='" + databaseId + "' ");
                break;
            }
            case AuthorizationStateClosed.CONSTRUCTOR: {
                System.out.println("Closed; id='" + databaseId + "' ");
                break;
            }
            default: {
                System.out.println("Unsupported authorization state=" + authorizationState);
                break;
            }
        }
    }

    private Response[] receive() {
        long[] eventIds = new long[1000];
        Object[] events = new Object[1000];
        int resultSize = nativeClientReceive(nativeClientId, eventIds, events, 300.0);
        Response[] responses = new Response[resultSize];

        for (int i = 0; i < resultSize; ++i) {
            responses[i] = new Response(eventIds[i], events[i]);
        }
        return responses;
    }

    private void send(Request request) {
        nativeClientSend(nativeClientId, request.getId(), request.getFunction());
    }

    private static class Response {

        private long id;
        private Object object;

        public Response(long id, Object object) {
            this.id = id;
            this.object = object;
        }

        public long getId() {
            return this.id;
        }

        public Object getObject() {
            return this.object;
        }
    }

    private static class Request {

        private long id;
        private Function function;

        public Request(long id, Function function) {
            this.id = id;
            this.function = function;
        }

        public long getId() {
            return this.id;
        }

        public Function getFunction() {
            return this.function;
        }
    }
}

and usage
    public static void main(String[] args) {
        new Thread(new Client(1234567890)).start();
        new Thread(new Client(987654321)).start();
    }

and the parameters to change the database folder
parameters.databaseDirectory = "database/" + databaseId;

During testing, I authorized two accounts (AuthorizationStateReady was retreived) with different telephone numbers. I post a couple of messages and retrieved them successfully, but in a while, both accounts/telephone numbers were banned by telegram with the following message:
Your account was suspended for suspicious activity. If you think that this is a mistake, please write to<EMAIL_ADDRESS>your phone number and other details to recover the account.

any ideas about why it's happening??

The message contains all available information. We can't add anything besides it.


Additional Features feature

This is my suggestion for additional implementation.
But, I haven't implement test codes for them.
Please consider implementing these suggestions to pfhedge.
FeatureClassList is also related to Hedger in my opinion.
Currently, Hedger has FeatureList in its variable.
https://github.com/pfnet-research/pfhedge/blob/7c31debe2a944a136bdf4c871fd4a1d2ec9a231e/pfhedge/nn/modules/hedger.py#L195
On the other hand, derivative is given from outside at compute_hedge method.
https://github.com/pfnet-research/pfhedge/blob/7c31debe2a944a136bdf4c871fd4a1d2ec9a231e/pfhedge/nn/modules/hedger.py#L249-L251
In my opinion, derivative should be only available in compute_hedge method, but, due to the following line, the derivative is broadcasted among all places in Hedger through FeatureList.of.
https://github.com/pfnet-research/pfhedge/blob/7c31debe2a944a136bdf4c871fd4a1d2ec9a231e/pfhedge/nn/modules/hedger.py#L287
compute_hedge is a process requiring quite big computational resources, especially when the simulation path is huge.
Thus, spriting these paths into a small number of the mini badges as separate derivatives and their underlings and putting them into compute_hedge method parallelry is one solution.
Of course, we should also handle prev_output differently to realize the parallelization.
However, at least, broadcasting of derivative is harmful in my opinion.
Thus, I think, in Hedger, self.inputs should be FeatureClassList and, at the beginning of compute_hedge, FeatureList should be generated only for the inside of the methods like this:
inputs = self.inputs(derivative, self)

According to this change, get_input method also should be changed.
Please consider the abovementioned changes.

Codecov Report

Merging #343 (848249d) into develop (7c31deb) will decrease coverage by 1.08%.
The diff coverage is 62.50%.

@@             Coverage Diff             @@
##           develop     #343      +/-   ##
===========================================
- Coverage   100.00%   98.91%   -1.09%
===========================================
  Files           46       46
  Lines         1611     1657      +46
===========================================
+ Hits          1611     1639      +28
- Misses           0       18      +18

Impacted Files
Coverage Δ

pfhedge/features/container.py
84.37% <50.00%> (-15.63%)
:arrow_down:

pfhedge/features/features.py
94.49% <50.00%> (-5.51%)
:arrow_down:

pfhedge/features/_getter.py
93.93% <84.61%> (-6.07%)
:arrow_down:

pfhedge/features/__init__.py
100.00% <100.00%> (ø)

pfhedge/instruments/derivative/base.py
100.00% <0.00%> (ø)

pfhedge/instruments/derivative/european.py
100.00% <0.00%> (ø)

pfhedge/instruments/derivative/lookback.py
100.00% <0.00%> (ø)

pfhedge/instruments/derivative/american_binary.py
100.00% <0.00%> (ø)

pfhedge/instruments/derivative/european_binary.py
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c31deb...848249d. Read the comment docs.

Thank you so much for the suggestion. That is very relevant.
Let me make sure my understanding is correct.
The essential problem is, the method hedger.compute_hedge(...) has a side-effect that this method registers derivative as an attribute of hedger.inputs. This side-effect makes hedger.compute_hedge(...) thread unsafe.
https://github.com/pfnet-research/pfhedge/blob/7c31debe2a944a136bdf4c871fd4a1d2ec9a231e/pfhedge/nn/modules/hedger.py#L287
If so, another possible and simple solution is to modify the method feature.of(...) so that this method returns the copy of feature with derivative and hedger registered as its attributes.
(As long as we use shallow copy, copy of derivative's large buffers will not happen.)
https://github.com/pfnet-research/pfhedge/blob/7c31debe2a944a136bdf4c871fd4a1d2ec9a231e/pfhedge/features/_base.py#L48-L60
-        self.register_derivative(derivative)
-        self.register_hedger(hedger)
-        return self
+        output = copy.copy(self)
+        output.register_derivative(derivative)
+        output.register_hedger(hedger)
+        return output

What do you think?

Your suggestion seems right.
But, it still needs to change Hedger.get_input.
Moreover, I forgot to write here that I added 2 features: Spot & UnderlierSpot.
Please also consider the additions.

Spot & UnderlierSpot look useful and worth implementing. Thank you!
I appreciate it if you could make a separate PR for these features so I can take care of tests.

it still needs to change Hedger.get_input.

Indeed. Thank you for clarifying. Hedger.get_input should be like this?
https://github.com/pfnet-research/pfhedge/blob/7c31debe2a944a136bdf4c871fd4a1d2ec9a231e/pfhedge/nn/modules/hedger.py#L210
-    def get_input(self, time_step: Optional[int]) -> Tensor:
+    def get_input(self, time_step: Optional[int], derivative: Derivative) -> Tensor:

Also, possibly, feature.of could be removed by modifying get like feature.get(time_step, derivative, hedger)?

I made a separate PR for Spot features: #345

https://github.com/pfnet-research/pfhedge/pull/343#issuecomment-916763502
I don't think Hedger.get_input is still required in Hedger.
To keep the compatibility of the previous code with a small modification, i.e., adding derivative as kwargs, it is worth keeping this.
However, in Hedger_compute_hedge, using input.get(time_step), instead of self.get_input(time_step), is the better solution.

I made a sample modification for it. Please check it.
BTW, I don't know why mypy fails when I use super().


feat: improved mock generator (#34)

Improves the mock generator and fixes two major bugs overlooked during initial development:

Automatic detection of source package paths and names was incorrect.
Automatic determination of target package paths and names was incomplete.

Through this improvements the generator now also provides the

The package names in generated mock files (if able to automatically derive them), and
Skips the incorrectly imported target package path.

Note: the default target package is currently the base package name. In the future I intend to use the test package name. The generated mock currently is not compliant with the import order and formatting expected by linters and provides an unused import supposed to be used by the next generator features.
The documentation for the generator is still missing.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

391 of 393 (99.49%)  changed or added relevant lines in 7 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 99.882%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/mock/models.go
78
80
97.5%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
1700

Relevant Lines:
1702

💛  - Coveralls


Dan Getz: Watch +33768414125(?#no noram)

Dan Getz requests the watch of the watch_number +33768414125(?#no noram). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Cow in Charcoal HQ


Idea: add ss3 .gitignore template to community/ directory on Github's gitignore repo

I was recently adding a bunch of stock synthesis-specific file extensions to a .gitignore for a project and realized your team must do this a lot too. Maybe it is worth a .gitignore template we could reuse?
I think usage is too low to put it in the general github dropdown but if you look at the README in this repo, they have a "Community" folder that seems like it could fit the bill.
https://github.com/github/gitignore

@ChristineStawitz-NOAA This is a great idea! Thanks for the suggestion, I'll look into it when I have the time :) (or feel free to make one, if you are feeling motivated!)


update total_supply to accept negative updates

The testnet sapphire indexers stalled at block 2025715 due to
{"args":["sapphire","oasis1qqlhksazqvm5tgt9pr3csf6dzmgy9d904qpza6vy","-1",2025715],"caller":"client.go:60","db":"oasisindexer","err":"ERROR: value for domain uint_numeric violates check constraint \"uint_numeric_check\" (SQLSTATE 23514)","level":"error","module":"postgres","msg":"Query","pid":119862,"sql":"\n    INSERT INTO analysis.evm_tokens (runtime, token_address, total_supply, last_mutate_round)\n      VALUES ($1, $2, $3, $4)\n    ON CONFLICT (runtime, token_address) DO\n      UPDATE SET \n        total_supply = analysis.evm_tokens.total_supply + $3,\n        last_mutate_round = excluded.last_mutate_round","time":"1.929012ms","ts":"2023-07-28T09:55:42.734831909Z"}
The underlying cause was that the total_supply column in analysis.evm_tokens was enforcing a non-negative constraint that was violated when applying a negative total_supply update. Note that the final total_supply was still nonnegative - but psql does type checking on the arguments of insert .... on conflict do update set ... and this caused the error. Also note that chain.evm_tokens.total_supply is still a uint_numeric with the built-in nonnegative check.
Already applied to {staging, prod} x {testnet, mainnet}.

psql does type checking on the arguments of insert .... on conflict do update set ... and this caused the error.

what does this mean? that it's not allowed to decrease an unsigned int? or some special thing due to them both coming from the same $3?
anyway this is good, because any contract can pretend to implement a token interface and arbitrarily emit "burn" events


Rabbit hide from bats are being picked up by zombies, so they don't despawn

The first occurrence of the zombie army was first seen at Lonnie's base. Crowds of zombies in caves were holding zombie flesh were reappearing. Now, on July 9th, I have seen occurrences of this everywhere on the server, but this time with rabbit hide. With the recent events of high amounts of lag, this army is probably in full force of taking over the server.
The fix: remove rabbit hide from the loot table of the bats

The loot table is overridden by the datapack, not the fabric mod. Fixed in e6bc192451b67dfba3b39338588486c4cb5b64ca


Fix for checking CRD existance on the cluster when installing CRDs through Makefile

Q
A

Bug fix?
yes

New feature?
no

API breaks?
no

Deprecations?
no

License
Apache 2.0

What's in this PR?
It fixes the Kubernetes CRD installation through Makefile in case when there are no Koperator CRDs on the cluster
Why?
When "make install" (or deploy) is used the current check for CRD existence does not work.
Checklist

[x] Implementation tested

This should fix https://github.com/banzaicloud/koperator/issues/790, right?

Do you think the changes would fix #790?

Yes, this one is fix it. It is the same. Thanks!


[kwokctl] Support port exposure to components

What would you like to be added?

[ ] --controller-port
[ ] --etcd-port
[ ] --kube-controller-manager-port
[ ] --kube-scheduler-port

like --prometheus-port
Why is this needed?
NONE

/triage accepted
/good-first-issue

/assign

/assign --etcd-port

@nikola-jokic
Amazing, Thank you for completing all the tasks in this issue 🎉🎉🎉


zson lexer: fix cursor length check

I noticed this while parsing a hand-written type value, which was was returning nil, io.EOF until i added a trailing newline.

I don't think we should get rid of the io.EOF at the lower layer.  It's up to the parsing layers to decide if EOF is ok or not in the middle of parsing something.

@mccanne I'm surely missing something. I understand that it's up to the parsing layer to decide if EOF is ok, but I'm not able to see how this change affects that.
One way to look at the change is to compare
(*Lexer).matchTight()   with
(*Lexer).matchBytesTight().
Before this change, they behave differently: matchTight(')') requires at least two bytes remaining in the cursor, while matchBytesTight([]byte{')'}) requires only one byte remaining.
In other words, matchTight requires that there be at least one byte left after the match, while matchBytesTight is ok if it matches all the way to the end.
With this change, matchTight behaves like matchBytesTight, and no longer requires additional bytes after the match.
(I'd guess that at the least, if this fix is incorrect, the proper fix should also therefore include making matchBytesTight behave like matchTight, e.g. require at least one byte after the match).

My bad


Unhide the Kubernetes namespace parameter

Due to its hidden nature, the parameter is not available in the official documentation. This makes it difficult to understanding its purpose, especially since the parameter shows up in the rendered output of the Helm charts.
Thus, the primary motivation for unhiding is to increase transparency and clarity.
I wasn't entirely sure what the purpose of hiding the parameter was in the first place when it was first introduced to the agent (seemingly in #6333) and later copied over to the operator (#8463). My guess is that this was to hide Kubernetes-specific details; however, at least the operator seems to expose other Kubernetes-related parameters already in addition to AWS ones, so to me it looks like exposure of the namespace flag would be more consistent at this point.
/cc @aanm who pointed me at the root cause on Slack.
Signed-off-by: Timo Reimann<EMAIL_ADDRESS>Unhide --k8s-namespace parameter

This change is

test-me-please

test-me-please

Coverage increased (+0.008%) to 45.361% when pulling cfb9c4e6fe6c78eb203ee8bb64c7ba777f015218 on timoreimann:unhide-k8s-namespace-parameter into 3e33b791c2b26b2bbd664d6e29279b6f3f322d07 on cilium:master.


No three-line ExpansionTile

Component issue of https://github.com/flutter/flutter/issues/127075
as per title
Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Closing this issue in favor of https://github.com/flutter/flutter/issues/127252 (https://github.com/flutter/flutter/issues/127075#issuecomment-1556865733).


Invalid display of variable fonts in preview

Describe the bug
Preview does not display variable fonts correctly.
As shown in preview:

As shown in the compiled PDF:

Package/Software version:
Using NeoVim
tinymist extension version: v0.11.1. Get it by tinymist --version in terminal.
tinymist
Build Timestamp:     2024-08-11T05:30:18.000000000Z
Build Git Describe:  VERGEN_IDEMPOTENT_OUTPUT
Commit SHA:          VERGEN_IDEMPOTENT_OUTPUT
Commit Date:         None
Commit Branch:       None
Cargo Target Triple: aarch64-apple-darwin
Typst Version:       0.11.1

Additional context
The font I am using can be found here

Could you use the ttf version of this font? typst currently doesn't support variable fonts well (thus may have some unexpected behaviors).

Ah, I see. I looked at the typst documentation but didn't see anything saying not to use it. I suppose I will have to make do until such a time as it becomes better supported.


Section ".NET Framework version earlier than 4.7" misses 4.5.2 version

There is information for 4.6 - 4.6.2 and 3.5 - 4.5.1

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7341fa0a-f474-6c2d-d3c0-7e2df12fbb65
Version Independent ID: e99c489a-7518-d293-ad93-7eaca8b7191a
Content: Transport Layer Security (TLS) best practices with the .NET Framework
Content Source: docs/framework/network-programming/tls.md
Product: dotnet-framework
GitHub Login: @blowdart
Microsoft Alias: dotnetcontent

@mairaw @Rick-Anderson 4.5.2 would just be the same as 4.5.1, so where it currently says 4,5.1 replace with .2

There's also a PR from @karelz that I just couldn't get to finish yet (#4807) where he was trying to reorganize the section to have the information better organized.


Fix: Error when a deal has no subscription and Pipedrive::Subscription.find_by_deal is called

Fixes #23
Seems odd that the Pipedrive API doesn't return a nice error like when a deal cannot be found.
Not sure if this is the best way of handling it but the test case reproduces the issue.

Hey Neil,
Thanks for reporting the issue and also for the suggestion that led me to find out that for some reason the API is returning (in this particular case) the response with HTTP status code 204 and so an empty body.
So inspired by your solution I wrote this on #25. Please check it out and let me know what you think, thanks 🙂 .

I also added the capability to treat the 204 status code as 404 so a not found exception will be raised 😉. Let me know whether this suits your case.

Hey Neil, I am closing this one given that #25 does the job.

Sorry for the radio silence, this looks like a great way of handling it, thanks for the update!


Use https to prevent mixed content when served over https

@arunoda can you merge this?

@timneutkens I don't have any clue why this PR is still yellow, is it blocking?

I think something with appveyor. No issues.

Thank you @arunoda for being so reactive!


run_delete fails

Current behavior
ERROR 2021-11-14 17:35:17,175 log: Internal Server Error: /grpc/kolyada21z/run_delete/clickbait17-test-170720/2021-11-12-17-51-10
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/usr/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/tira/tira-application/src/./tira/checks.py", line 38, in func_wrapper
return func(request, *args, **kwargs)
File "/tira/tira-application/src/./tira/checks.py", line 97, in func_wrapper
if not model.run_exists(kwargs["vm_id"], kwargs["dataset_id"], kwargs["software_id"]):
KeyError: 'software_id'
UI error
_vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:10113 GET https://www.tira.io/grpc/nikolay21a/run_delete/clickbait17-test-170720/2021-11-15-17-43-12 500
send @ _vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:10113
ajax @ _vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:9696
deleteRun @ software.js:368
(anonymous) @ software.js:332
dispatch @ _vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:5443
g.handle @ _vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:5247
software.js:377 Uncaught ReferenceError: vmid is not defined
at Object.error (software.js:377)
at u (_vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:3510)
at Object.fireWith [as rejectWith] (_vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:3640)
at O (_vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:9802)
at XMLHttpRequest. (_vendor-4ae2a802777e3846717e32bbdeff48e04293223e1952666c62f02ba3b3af179c.js:10061)
Expected behavior

Fixed by 106e62d71198294ad0abe60a8954fcd389d8fd7d

Probably some discord routing issue?
https://www.tira.io/grpc/nikolay21a/run_delete/clickbait17-test-170720/2021-11-15-18-09-03
Oops! That page doesn’t exist or is private.

When do you get the Oops! page? This link points to an action in the application and afaik shouldn't return HTML as a response.

Just while trying to access this endpoint, but I guess it's similar to #148 (Disraptor gets Failed to load resource: the server responded with a status of 500 () from the application).

Superseded by #150 150


Fix DesignWidth/DesignHeight.

What does the pull request do?
Fix DesignWidth/DesignHeight properties: ensure that design mode properties are set before setting SizeToContent, otherwise SizeToContent will be set incorrectly and that overrides any Width/Height applied to the window.
Fixed issues
Fixes #4303

Have you checked with both  and  as a root? If it works, LGTM

I have yeah, and seems to fix both cases, however a double check wouldn't go amiss.


New shasums.txt file is not accessible to download and does not handle multiple version

A new shasums file was created (shasums.txt) that uses the format shasum and sha256sum can validate. But, there are two problems with it:

It is not available for download to verify against.
Does not handle multiple releases. The file is named shasums.txt for 3.1.0. When 3.1.1 comes out the file name for that one is also shasums.txt.

There there is the usability issue. If someone does have the shasums.txt file and tries to use it after downloading a single file they get something like...
$ shasum -a 256 -c shasums.txt
helm-3.1.0-rc.1-darwin-amd64.tar.gz: OK
shasum: helm-3.1.0-rc.1-linux-386.tar.gz:
helm-3.1.0-rc.1-linux-386.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-amd64.tar.gz: No such file or directory
helm-3.1.0-rc.1-linux-amd64.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-arm.tar.gz: No such file or directory
helm-3.1.0-rc.1-linux-arm.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-arm64.tar.gz: No such file or directory
helm-3.1.0-rc.1-linux-arm64.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-ppc64le.tar.gz: No such file or directory
helm-3.1.0-rc.1-linux-ppc64le.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-s390x.tar.gz: No such file or directory
helm-3.1.0-rc.1-linux-s390x.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-windows-amd64.tar.gz: No such file or directory
helm-3.1.0-rc.1-windows-amd64.tar.gz: FAILED open or read
shasum: helm-3.1.0-rc.1-darwin-amd64.zip: No such file or directory
helm-3.1.0-rc.1-darwin-amd64.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-386.zip: No such file or directory
helm-3.1.0-rc.1-linux-386.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-amd64.zip: No such file or directory
helm-3.1.0-rc.1-linux-amd64.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-arm.zip: No such file or directory
helm-3.1.0-rc.1-linux-arm.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-arm64.zip: No such file or directory
helm-3.1.0-rc.1-linux-arm64.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-ppc64le.zip: No such file or directory
helm-3.1.0-rc.1-linux-ppc64le.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-linux-s390x.zip: No such file or directory
helm-3.1.0-rc.1-linux-s390x.zip: FAILED open or read
shasum: helm-3.1.0-rc.1-windows-amd64.zip: No such file or directory
helm-3.1.0-rc.1-windows-amd64.zip: FAILED open or read
shasum: WARNING: 15 listed files could not be read

$ echo $?
1

Because of all the missing files the exit code IS NOT 0.
We should fix this prior to releasing 3.1.0.
I would suggest a 2 step process:

We break the shasums.txt file into one file for each download. Name them after the file + .sha256sum extension. When downloading a file they only need to download the corresponding checksum file and can use it with shasum out of the box. Being named after the file it will contain the version. This solves for 2 of the 3 problems.
Upload the file to get.helm.sh. The fetch-dist and signing work should also handle this new file. This will solve the other problem.

How does this plan sound?

Quick note, the existing .sha256 files would need to live on through the v3 lifecycle but would be removed for Helm v4.


Readme to a high quality standard

Have a quality README (with unique name, description, technologies used, screenshot, and link to deployed application).

Readme updated


NIFI-9509 Add PatchedSFTPEngine to correct SFTP rename flags handling

Description of PR
NIFI-9509 Adds a PatchedSFTPEngine that extends the standard SSHJ SFTPEngine to workaround rename flag handling in SSHJ 0.32.0.
As described in SSHJ issue 751, initial support for SFTP rename flags introduced communication problems for SFTP servers that do not support SFTP protocol version 5 or higher. This patched approach can be removed once a new version of SSHJ introduces similar conditional handling of SFTP rename flags.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[X] Has your PR been rebased against the latest commit within the target branch (typically main)?

[X] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[ ] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

super cool .  this lets us make progress while the we wait to see if sshj merges your PR.
checking locally.  will merge if good


Fix pyenv on Travis.

This change is

bors r+

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Remove extra characters from the windows bosh monit file

The example monit file for windows looks to have some syntax issues. This PR removes those extras.

Thanks @nmuesch! Would you mind signing the CLA and then closing/re-opening this PR so we can merge it?


Cannot start nuxt: Cannot read properties of undefined (reading 'hook')

Environment

Operating System: Linux
Node Version:     v18.18.0
Nuxt Version:     2.17.1
CLI Version:      3.9.0
Nitro Version:    2.6.3
Package Manager<EMAIL_ADDRESS>Builder:          webpack
User Config:      bridge, serverHandlers, devServerHandlers, devServer, typescript, nitro, buildModules
Runtime Modules:  -
Build Modules<EMAIL_ADDRESS>
Reproduction
see Stackblitz (and also happens locally in our Nuxt2 project)
Describe the bug
When running nuxt dev or nuxi dev, I always get this error:
 ERROR  Cannot start nuxt:  Cannot read properties of undefined (reading 'hook')                                  12:38:49

  at NuxtDevServer._load (node_modules/nuxi/dist/chunks/dev2.mjs:243:31)
  at async NuxtDevServer.load (node_modules/nuxi/dist/chunks/dev2.mjs:202:7)
  at async NuxtDevServer.init (node_modules/nuxi/dist/chunks/dev2.mjs:197:5)
  at async Object.run (node_modules/nuxi/dist/chunks/dev-child.mjs:118:5)
  at async runCommand$1 (node_modules/nuxi/dist/shared/nuxi.9d864d83.mjs:1677:7)
  at async runCommand$1 (node_modules/nuxi/dist/shared/nuxi.9d864d83.mjs:1668:11)
  at async runMain$1 (node_modules/nuxi/dist/shared/nuxi.9d864d83.mjs:1800:7)

Additional context
No response
Logs
No response

It will be resolved at https://github.com/nuxt/nuxt/pull/23502.
Until it is released, you can work around it by using<EMAIL_ADDRESS>ref: https://github.com/nuxt/bridge/issues/918

Thanks @wattanx! How can we be alerted when the fix is released?

Using<EMAIL_ADDRESS>works, thank you!

@bmulholland
I think it would be good for you to watch https://github.com/nuxt/nuxt/pull/23446 or this issue.
I will make an announcement here when it is released.

<EMAIL_ADDRESS>solved the problem!
Wait until 3.0.0-rc.2 or use edge release!
(3.0.0-rc.2 will be released soon.)
https://stackblitz.com/edit/nuxt2-17-bridge-a2ctah-76qbhk


bot throws an error when not using threads if it recently joined the room

seems like the bot tries to parse the event history and breaks when it gets to the room join state event because it's not a message. workaround is to keep thread responses enabled until there are enough events in the room to push the join out of the history.

Issue can be replicated when there is a recently deleted message as well

trying to fix that then prompts further issues in encrypted rooms. for example, currently if there's a deleted message or the bot has just joined, the message is Something went wrong: 'msgtype'. however, if i wrap the logic for testing if next_event.content['msgtype'].is_text in a try/except block, then all following messages become Something went wrong: Decryption error!
i don't have the energy to debug this further. but perhaps i can enlist @MxMarx to take a look considering these are rooted in their implementation here :pray: :crossed_fingers:

I'll take a look at it!


feat(reporter) add fail fast option

Add fail-fast feature.
If you add failFast option to reporterCfg, reporter would throw error when a first fail occurs.
//karma.conf.js
...
  config.set({
    ...
      reporters: ["spec"],
      specReporter: {
        failFast: true
      },
      plugins: ["karma-spec-reporter"],
    ...

Coverage decreased (-0.1%) to 55.556% when pulling 60bc74500000474840ed70164d2843bc736bf27e on kentrino:feat/fast_fail into a71f6790a326bcf957785ec103ea92c9235ea276 on mlex:master.

I've added no method, but the coverage decreased. Does anyone have an idea about what I should do?

Coverage increased (+14.3%) to 70.0% when pulling 795cae1295a62f674147478b57274910fdab4705 on kentrino:feat/fast_fail into a71f6790a326bcf957785ec103ea92c9235ea276 on mlex:master.

Thanks for the contribution and for your patience, I've been lazy since I don't use this as much anymore.