The need for maintaining standard Market1501 evaluation set splits?

Hi, Tong Xiao. I found that your splitting of the evaluation set of Market1501 is different from the standard manner.
In your splits, the statistics look as follows:
Market1501 dataset loaded
  subset   | # ids | # images
  ---------------------------
  train    |   651 |    11387
  val      |   100 |     1549
  trainval |   751 |    12936
  query    |   750 |    16483
  gallery  |   751 |    19281

While the standard one looks like:
Market1501 dataset loaded
  subset   | # ids | # images
  ---------------------------
  train    |   651 |    11387
  val      |   100 |     1549
  trainval |   751 |    12936
  query    |   750 |     3368
  gallery  |   751 |    15913

The difference resides in the number of query and gallery images. In your partitioning, for each query id, the query set gathers all its images. While in the standard partitioning, for each query id, only some selected images (at most one image for one camera) are placed in the query set.
It makes some difference in the evaluation results. In the triplet loss example, using the same trained model, the open-reid way of partitioning gives results:
Mean AP: 67.6%
CMC Scores    allshots      cuhk03  market1501
  top-1          42.7%       70.1%       84.5%
  top-5          59.3%       91.0%       94.0%
  top-10         67.2%       95.0%       96.5%

while on the standard eval splits, it gives:
Mean AP: 66.5%
CMC Scores    allshots      cuhk03  market1501
  top-1          44.0%       70.0%       82.5%
  top-5          61.9%       91.1%       93.0%
  top-10         69.9%       95.1%       95.7%

May I ask whether it's just your original intention or it's a little mistake?
After diving into your code, I made some modification to the reid/datasets/market1501.py and let it do the standard splitting:
from __future__ import print_function, absolute_import
import os.path as osp
import numpy as np

from ..utils.data import Dataset
from ..utils.osutils import mkdir_if_missing
from ..utils.serialization import read_json
from ..utils.serialization import write_json

########################
# Added
def _pluck(identities, indices, relabel=False):
  """Extract im names of given pids.
  Args:
    identities: containing im names
    indices: pids
    relabel: whether to transform pids to classification labels
  """
  ret = []
  for index, pid in enumerate(indices):
    pid_images = identities[pid]
    for camid, cam_images in enumerate(pid_images):
      for fname in cam_images:
        name = osp.splitext(fname)[0]
        x, y, _ = map(int, name.split('_'))
        assert pid == x and camid == y
        if relabel:
          ret.append((fname, index, camid))
        else:
          ret.append((fname, pid, camid))
  return ret
########################

class Market1501(Dataset):
  url = 'https://drive.google.com/file/d/0B8-rUzbwVRk0c054eEozWG9COHM/view'
  md5 = '65005ab7d12ec1c44de4eeafe813e68a'

  def __init__(self, root, split_id=0, num_val=100, download=True):
    super(Market1501, self).__init__(root, split_id=split_id)

    if download:
      self.download()

    if not self._check_integrity():
      raise RuntimeError("Dataset not found or corrupted. " +
                         "You can use download=True to download it.")

    self.load(num_val)

  def download(self):
    if self._check_integrity():
      print("Files already downloaded and verified")
      return

    import re
    import hashlib
    import shutil
    from glob import glob
    from zipfile import ZipFile

    raw_dir = osp.join(self.root, 'raw')
    mkdir_if_missing(raw_dir)

    # Download the raw zip file
    fpath = osp.join(raw_dir, 'Market-1501-v15.09.15.zip')
    if osp.isfile(fpath) and \
            hashlib.md5(open(fpath, 'rb').read()).hexdigest() == self.md5:
      print("Using downloaded file: " + fpath)
    else:
      raise RuntimeError("Please download the dataset manually from {} "
                         "to {}".format(self.url, fpath))

    # Extract the file
    exdir = osp.join(raw_dir, 'Market-1501-v15.09.15')
    if not osp.isdir(exdir):
      print("Extracting zip file")
      with ZipFile(fpath) as z:
        z.extractall(path=raw_dir)

    # Format
    images_dir = osp.join(self.root, 'images')
    mkdir_if_missing(images_dir)

    # 1501 identities (+1 for background) with 6 camera views each
    identities = [[[] for _ in range(6)] for _ in range(1502)]
    def register(subdir, pattern=re.compile(r'([-\d]+)_c(\d)')):
      fnames = [] ######### Added. Names of images in new dir.
      fpaths = sorted(glob(osp.join(exdir, subdir, '*.jpg')))
      pids = set()
      for fpath in fpaths:
        fname = osp.basename(fpath)
        pid, cam = map(int, pattern.search(fname).groups())
        if pid == -1: continue  # junk images are just ignored
        assert 0 <= pid <= 1501  # pid == 0 means background
        assert 1 <= cam <= 6
        cam -= 1
        pids.add(pid)
        fname = ('{:08d}_{:02d}_{:04d}.jpg'
                 .format(pid, cam, len(identities[pid][cam])))
        identities[pid][cam].append(fname)
        shutil.copy(fpath, osp.join(images_dir, fname))
        fnames.append(fname) ######### Added
      return pids, fnames

    trainval_pids, _ = register('bounding_box_train')
    gallery_pids, gallery_fnames = register('bounding_box_test')
    query_pids, query_fnames = register('query')
    assert query_pids <= gallery_pids
    assert trainval_pids.isdisjoint(gallery_pids)

    # Save meta information into a json file
    meta = {'name': 'Market1501', 'shot': 'multiple', 'num_cameras': 6,
            'identities': identities,
            'query_fnames': query_fnames, ######### Added
            'gallery_fnames': gallery_fnames} ######### Added
    write_json(meta, osp.join(self.root, 'meta.json'))

    # Save the only training / test split
    splits = [{
      'trainval': sorted(list(trainval_pids)),
      'query': sorted(list(query_pids)),
      'gallery': sorted(list(gallery_pids))}]
    write_json(splits, osp.join(self.root, 'splits.json'))

  ########################
  # Added
  def load(self, num_val=0.3, verbose=True):
    splits = read_json(osp.join(self.root, 'splits.json'))
    if self.split_id >= len(splits):
      raise ValueError("split_id exceeds total splits {}"
                       .format(len(splits)))
    self.split = splits[self.split_id]

    # Randomly split train / val
    trainval_pids = np.asarray(self.split['trainval'])
    np.random.shuffle(trainval_pids)
    num = len(trainval_pids)
    if isinstance(num_val, float):
      num_val = int(round(num * num_val))
    if num_val >= num or num_val < 0:
      raise ValueError("num_val exceeds total identities {}"
                       .format(num))
    train_pids = sorted(trainval_pids[:-num_val])
    val_pids = sorted(trainval_pids[-num_val:])

    self.meta = read_json(osp.join(self.root, 'meta.json'))
    identities = self.meta['identities']

    self.train = _pluck(identities, train_pids, relabel=True)
    self.val = _pluck(identities, val_pids, relabel=True)
    self.trainval = _pluck(identities, trainval_pids, relabel=True)
    self.num_train_ids = len(train_pids)
    self.num_val_ids = len(val_pids)
    self.num_trainval_ids = len(trainval_pids)

    ##########
    # Added
    query_fnames = self.meta['query_fnames']
    gallery_fnames = self.meta['gallery_fnames']
    self.query = []
    for fname in query_fnames:
      name = osp.splitext(fname)[0]
      pid, cam, _ = map(int, name.split('_'))
      self.query.append((fname, pid, cam))
    self.gallery = []
    for fname in gallery_fnames:
      name = osp.splitext(fname)[0]
      pid, cam, _ = map(int, name.split('_'))
      self.gallery.append((fname, pid, cam))
    ##########

    if verbose:
      print(self.__class__.__name__, "dataset loaded")
      print("  subset   | # ids | # images")
      print("  ---------------------------")
      print("  train    | {:5d} | {:8d}"
            .format(self.num_train_ids, len(self.train)))
      print("  val      | {:5d} | {:8d}"
            .format(self.num_val_ids, len(self.val)))
      print("  trainval | {:5d} | {:8d}"
            .format(self.num_trainval_ids, len(self.trainval)))
      print("  query    | {:5d} | {:8d}"
            .format(len(self.split['query']), len(self.query)))
      print("  gallery  | {:5d} | {:8d}"
            .format(len(self.split['gallery']), len(self.gallery)))
  ########################

If it's your original intention, just ignore my issue, and IMHO, you may add some notes in the doc.
Thank you again for your code!

Thank you very much for pointing out this issue! For Market-1501 and DukeMTMC, our current code choose query according to the person IDs, rather than the images. This could have more query images than originally defined, and indeed causes a mismatch with the official evaluation settings. However, the difference between our evaluation setting and the official one is usually less than 1% mAP according to my test. And arguably splitting by person IDs is better IMHO.
Nevertheless, your solution is really good, but I think we might need to do a bit more for a new split protocol that works for all the datasets. I think one way is to store the filenames directly to split.json. I will soon figure out a way to use exactly the official settings on these two datasets. Thanks again for the suggestion!

Nice explanation, now it makes sense to me that more queries means better averaging and leads to more stable evaluation results.
And my quick and rough modification to the Market1501 class is only for my own verification (I used the open-reid API functions in another experimental project with standard Market1501 eval splits), so it's not universal at all :)
Thanks for your kind help!

Hi,
I'm trying to reproduce your benchmark on Market1501. For now, I successfully accomplished the Resnet50 both triplet and softmax, but when using inception module, I always get 10 percent lower than benchmark. By the way, I'm using my own way to read data, but I think it's fine, because it worked well with Resnet50 , my training parameters are like below:
args {
          weight_decay: 0.0005
          is_cuda: True
          workers: 4
          batch_size: 256
          height: 144
          epochs: 150
          epoch_step: 75
          width: 56
          seed: 1
          lr: 0.1(also test with 0.01)
          log_interval: 10
          gpus: 2,3
          val_interval: 50
          dataset: market1501
          features: 128
  }

Any suggestion?


hue and saturation imgaug assume BGR

hue and saturation imgaug assume BGR but ImageFromFile already fixes BGR -> RGB

btw, when i played with saturation augmentation it seemed like it was changing the colors (even when taking into account to BGR/RGB issue)

Maybe it's OK to move to RGB default to be consistent within dataflow/.


Updated helm charts don't appear in list without app recycle

Describe the bug
When new versions of a chart are available (and I've done a helm repo update to validate), the new chart versions do not show up in the list. Same is true of new charts in general. If I close and re-open the app, the new charts / versions show up.
To Reproduce
Steps to reproduce the behavior:

Open Lens and load the Charts tab
Push a new chart or version to a repo
click around a bit in lens and notice that the new chart or version isn't there
close lens and reopen it and notice it is there

Expected behavior
I expect the new chart or version to show up when I do a helm repo update, or for there to be some button inside the Lens GUI to flush cache / refresh list, etc.
Environment (please complete the following information):

Lens Version: 3.5.3
OS: Windows 10 v2004
Installation method (e.g. snap or AppImage in Linux): exe download/install

I got the same trouble.
Environment (please complete the following information):
Lens Version: 4.0.5
OS: Ubuntu 20.04.1
Installation method (e.g. snap or AppImage in Linux): AppImage

I got the same trouble.
Environment (please complete the following information):
Lens Version: 4.0.5
OS: Ubuntu 20.04.1
Installation method (e.g. snap or AppImage in Linux): AppImage

I actually can't make Lens update the versions at all, ie. even a restart of the application does not help.

Lens version: 4.1.2
OS: Windows 10 20H2
Installation method (e.g. snap or AppImage in Linux): exe download/install

I have the same problem.
It seems they are cached somewhere and not reloaded. It would be great if we could reload them somehow. One possibility would be a refresh button on the right side of the search-bar or a right click option menu inside the chart-list.
This is especially useful if you deploy your own charts and want to test them.

Related line is https://github.com/lensapp/lens/tree/67898152171cd8a9f6f927f12c47aa306105158c/src/main/helm/helm-chart-manager/ts#L70

Hey @Nokel81 funny I just checked out the repo to search for the line myself and then you posted it 2 min. later 😄
The line you found references a cacheFile. I could not find it saved in the lens folder %appdata%. Is it managed through a worker somewhere?
I got this path from helm-repo-manager.ts line 100:
cacheFilePath: `${this.helmEnv.HELM_REPOSITORY_CACHE}/${repo.name}-index.yaml`

Is it not saved within Lens but by helm itself, Lens merely reads it once per repo name.

Oh okay so when I update my helm repo it should update too?

I correct myself i did helm repo update and I can see my updated charts from my private registry. I restarted Lens and it still does not show the updated charts. Maybe I am missing something

That should have worked. Did you quit the tray icon too?

Nice Awesome!!! That helped for now! Thank you so much!

You are welcome, I am going to leave this open since we should periodically reload that file.

I have the same issue here.
Using microk8s, helm3. Closing the Tray Icon does not refresh the list of helm charts.
Only removing the cluster and re-adding seems to help right now

Same issue, I did a helm repo update locally and on the server, my new Helm Chart's version doesn't appear :(

Closing the Tray Icon does not refresh the list of helm charts.

Do you mean picking "Quit Lens"?

I am having this issue now, if I add a new repo it will only show up after reconnection to the cluster


Discord server invite not working

Apologise for writing this here but not sure how else to contact you fine folks. Unfortunately I cannot access the discord server. Have tried with on couple of laptops and phone. There shouldn't be any reason for me to get bounced as far as I am aware. Is there a constraint on the service currently or is likely something at my end.

This should have been fixed


[FIX] Allow web sdk to be called after prior requests error out

The issue here is: after a request results in an error, e.g. no network connection, web SDK refuses to take any additional incoming method calls.
The cause is that we didn't set req.onerror()
@dmitrig01

if there's an error init'ing, will it block all further calls? or will a subsequent call make the init retry?
can we get some tests around this?

also there are still some console.logs in here :-)

@dmitrig01
luckily for init we keep retrying (on apr/_r), and when the user reconnect the init can finish without an error
as for cross-platform, the unit tests all passed
And I removed the silly console logs

this is great
can we write another test that ensures the retry, i.e. if you call branch.init() and it fails, then you call branch.link(), the latter call will re-try the init call and then if that succeeds, call link?

After thinking it over, this PR on it's own should be fine - but let's go over it with @aaustin

:+1:


[docs-only] Fix web service envvar text

References: #7486 (feat: add config for disabling web extensions)
@JammingBen some questions:

As written in plural, how to separate multiple ID's?
By blank as usual?
As one needs to enter ID's, how does he be able to identify or get an ID?
We name Web extensions. Can you explain?

Knowing this, we can further improve the text.

As written in plural, how to separate multiple ID's? By blank as usual?

When provided via json, it's a normal list: ["extension1", "extension2"]. Honestly I don't know how that would work via env var, maybe @kobergj can give some insight? How does it work e.g. for WEB_OPTION_PREVIEW_FILE_MIMETYPES, which also is defined as a list of strings? Does it even work via env var?

As one needs to enter ID's, how does he be able to identify or get an ID?

Each Web extension is registered via an ID. There is no generic way how to get it, it depends on the extension. But usually it's right where the extension is being registered/defined in Web. See the search provider extension for the files app for example: https://github.com/owncloud/web/blob/master/packages/web-app-files/src/extensions.ts#L22

We name Web extensions. Can you explain?

A Web extension can basically by anything that extends the functionality of Web and lives outside its core. In most cases this will be a file editor, a file action or a search provider. There will be more extension types in the future though.

When provided via json, it's a normal list: ["extension1", "extension2"]. Honestly I don't know how that would work via env var, maybe @kobergj can give some insight? How does it work e.g. for WEB_OPTION_PREVIEW_FILE_MIMETYPES, which also is defined as a list of strings? Does it even work via env var?

The representation of a list through envvars is a comma separated list. WEB_OPTION_PREVIEW_FILE_MIMETYPES="extension1,extension2" would be interpreted as ["extension1", "extension2"] in json. As far as I can tell this is true for all "list" envvars we support.

I have updated the PR accoding the input from @kobergj and @JammingBen. Ready for review.

We possibly need another round as the way how multiple values for envvars is not clear, see: #7503

I fixed the conflict, corrected the envvar text accordig our discussion and squash all


Upgrade to selenium 2.44.0

Upgrade to selenium-java 2.44.0
Remove using deprecated (and now removed) method Proxies.extractFrom, see https://code.google.com/p/selenium/issues/detail?id=8088

+1

+1  - what andresriancho said.

Guys,
I'm trying to get a fix in place until an official release is made and am a little confused as to how to tell PhantomJs to use specific ghostdriver that contains the fix. I have grabbed the latest ghostdriver source from master and also downloaded the 1.2.1 fix mentioned by @asolntsev from here: http://mvnrepository.com/artifact/com.codeborne/phantomjsdriver/1.2.1.
I am using C# WebDriver bindings, running a selenium server 2.44.0 and PhantomJs 1.9.8. From my understanding PhantomJs has an old ghostdriver embedded into it that doesn't contain a fix for this issue, but you can override that using the phantomjs.ghostdriver.path option (i read this here: https://github.com/detro/ghostdriver/blob/master/README.md). I have tried using the following code to specify the custom ghostdriver path:
capabilities.SetCapability("phantomjs.ghostdriver.path", @"C:\ghostdriver-master\src\main.js");
I have tried many different options as well as pointing to the phantomjsdriver-1.2.1.jar file that i downloaded and still can't get it to work.
Do you have any suggestions on how i can get PhantomJs to use the updated ghostdriver that has this fix implemented? (i think im a little confused on exactly how to use this setting)
Any help would be greatly appreciated, thanks!

+1
Is there a problem with releasing this? I am still waiting desperately.
BTW: Thanks you all for the work you put into this. My work could not be with selenium, phantomjs and the awesome ghostdriver.

+1 What is the status of this issue?

+1 for releasing this.

Just as future note, Arquillian Drone and Graphene is migrating to com.codeborne fork. Selenium 2.45.0 was released recently and this issue takes 6 month to be released. Thanks @asolntsev for making that possible!

+1

+1

+1

+1 :)

I'm not sure if I can trust @asolntsev. @kpiwko vouches for him, claiming the arquillian stuff is switching to his fork. But I don't know of @kpiwko either. You can verify this claim following transitive dependencies:

https://github.com/arquillian/arquillian-graphene/blob/87dd6253329ed55801ebcb955431b89aba9645bf/pom.xml

https://github.com/arquillian/arquillian-extension-drone/blob/8b54eb8915d217e1028fc82631fd966903c27fa2/drone-bom/pom.xml

https://github.com/qa/arquillian-phantom-driver/blob/c86f2749f8a99e1ed4233049c211a691d71d38ef/pom.xml

@kpiwko is apparently a RedHat employee, and you can see his contributions on this site. Since his claim checks out, it seems reasonable to transitively trust @asolntsev.
Just thought I'd mention this in case anyone else was a little hesitant to start using the @asolntsev fork.

Hi Djeikyb!
What i like about open-source development is that you don't need to
trust me. The only think in the World you can trust is the code. You
can just compare original phantomjs code and my fork and verify that
there is no more difference but the few required lines of code.
Sincerely yours,
@asolntsev
2015-04-16 16:06 GMT+08:00, djeikyb<EMAIL_ADDRESS>
I'm not sure if I can trust @asolntsev. @kpiwko vouches for him, claiming
the arquillian stuff is switching to his fork. But I don't know of @kpiwko
either. You can verify this claim following transitive dependencies:

https://github.com/arquillian/arquillian-graphene/blob/87dd6253329ed55801ebcb955431b89aba9645bf/pom.xml

https://github.com/arquillian/arquillian-extension-drone/blob/8b54eb8915d217e1028fc82631fd966903c27fa2/drone-bom/pom.xml

https://github.com/qa/arquillian-phantom-driver/blob/c86f2749f8a99e1ed4233049c211a691d71d38ef/pom.xml
@kpiwko is apparently a RedHat employee, and you can see his contributions
on this site. Since his claim checks out, it seems reasonable to
transitively trust @asolntsev.
Just thought I'd mention this in case anyone else was a little hesitant to
start using the @asolntsev fork.

Reply to this email directly or view it on GitHub:
https://github.com/detro/ghostdriver/pull/399#issuecomment-93672368

--
Andrei Solntsev

@asolntsev do you have any knowledge about what actaully happened to this repo here? I mean why it is abandoned? Maybe there should be a message in the readme.me telling people that this repo is not maintained any more.

This repo doesn't look abandoned, @detro has made a commit 9hrs ago. There simply are no new releases for some reason...

That is good news! interesting timing for the first commit since November :) I would be more than glad if @detro continues his wonderful work!

Yes,
some months ago @detro wrote that he is not able to support the project anymore, and asked for volunteers. I guess he found somebody.
Sent from my HTC
----- Reply message -----
From: "luksch"<EMAIL_ADDRESS>To: "detro/ghostdriver"<EMAIL_ADDRESS>Cc: "Andrei Solntsev"<EMAIL_ADDRESS>Subject: [ghostdriver] Upgrade to selenium 2.44.0 (#399)
Date: Fri, Apr 17, 2015 02:01
@asolntsev do you have any knowledge about what actaully happened to this repo here? I mean why it is abandoned? Maybe there should be a message in the readme.me telling people that this repo is not maintained any more.
—
Reply to this email directly or view it on GitHub.

This would resolve so many headaches on my end!!! +1 to release this asap.

Just chiming in as others have done to see if a release is going to happen. If not, okay, but then let's make that abundantly clear to everyone. Incidentally, I came here via this ticket (https://github.com/detro/ghostdriver/issues/397). I'm using the solution provided there by @asolntsev and it's working fine.
Given that the "GhostDriver needs a proper maintainer" text is still on the readme, I'm guessing that a proper maintainer has not yet been found?


expand example on README

having never used passport until trying this project, I was unfamiliar with the basic concepts of passport and its Strategies. So I added some of the fundamentals to the examples in the readme.

repo not maintained, closing.


But how to add it to webthings-gateway?

Hi There,
I have set up webthings-gateway, as per the description on their GitHub page, however, after I enable your app on the phone, on port 8088, the gateway is not able to 'find' it, even if I add the URL with the phone's IP manually. The phone and gateway are on the same sub-net, they can ping each other. Is there anything else special to set up on android, maybe root (already have)? Or a special plugin or add-on needs to be installed on the gateway?
Thank you,

That should be everything, however it is possible that a recent update has broken the app, since the latest published version isn't running against the latest webthing library.

Hello,
Thank you for your quick answer! Yes, that might as well be the case, as I don't have any special config, eg. my wired network can 'reach' my wireless network, as both of them are on the same subnet, there are no special firewalls running on the server where the gateway is hosted, nor on my android devices. As a matter of fact I am using LineageOS on my 'lab' android mobiles, which are rooted, have no gapps, and installed webthingify trough f-droid by allowing 'incompatible' apps.
Thanks,

Issue related to  #11


Detect environment in .config() function?

Does this project have the ability to detect configuration within the Angular .config() function?
The goal of this is to, depending on the environment, set an Angular constant with the base URL for the API that the Angular app consumes.

Found that $environmentProvider.$get() will return an object with the current environment's details.  Thanks.


Can't add from some servers

I was able to add users from mastodon.world and scholar.social without issue, but when I try to add a user from mastodon.cloud or hcommons.social I get an error:
`There has been a critical error on this website. Please check your site admin email inbox for instructions.
Learn more about troubleshooting WordPress.`
The error from the email is:
`When seeking help with this issue, you may be asked for some of the following information:
WordPress version 6.1.1
Active theme: Rare Child (version 1.0.0)
Current plugin: ActivityPub (version 0.14.2)
PHP version 7.4.33
Error Details
An error of type E_ERROR was caused in line 121 of the file FOLDER/wp-content/plugins/activitypub/includes/functions.php. Error message: Uncaught Error: Object of class WP_Error could not be converted to string in FOLDER/wp-content/plugins/activitypub/includes/functions.php:121
Stack trace:
#0 FOLDER/wp-content/plugins/activitypub/integration/class-friends-feed-parser-activitypub.php(70): Activitypub\get_remote_metadata_by_actor()
#1 FOLDER/wp-content/plugins/friends/includes/class-feed.php(881): Activitypub\Friends_Feed_Parser_ActivityPub->update_feed_details()
#2 FOLDER/wp-content/plugins/friends/includes/class-admin.php(1579): Friends\Feed->discover_available_feeds()
#3 FOLDER/wp-content/plugins/friends/includes/class-admin.php(1780): Friends\Admin->process_admin_add_friend()
#4 FOLDER/wp-includes/class-wp-hook.php(308): Friends\Admin->render_admin_add_friend()
#5 FOLDER/wp-includes/class-wp-hook.php(332): WP_Hook->apply_filters()
#6 FOLDER/wp-includes/plugin.php(517): WP_Hook->do_action()
#7 FOLDER/wp-admin/admin.php(259): do_action()
#8 {mai
`
I don't know if the issue is with my setup or the servers I'm trying to connect to. 😎

Hmm, the plugin should not fail with a PHP fatal error ever. I will have a look, thanks for reporting.

@akirk is this problem also fixed with the latest version of the friends plugin?

It's actually a bug in the ActivityPub plugin and will be fixed with #213 which adds a if ( is_wp_error( $actor ) ) {.

oh! can you file a PR only for that change, so that we mabe can release a quick minor?

Ok, I created https://github.com/pfefferle/wordpress-activitypub/pull/220 with the fix.


Deformable DETR not fully end-to-end because of missing no_object class?

Hello,
I have another question. First I want to know if I understood the method for suppressing multiple detections and the used label loss correctly. In contrast to the original DETR, the Deformable DETR doesn't output no_object class. Thats why in training the target of all unmatched detections ist not a high score for no_object class, but a low score for every class. And thats why you use focal loss with binary cross entropy for the label loss.
My question is now, if the Deformable DETR is still fully end-to-end since I have to set a score threshold to filter out no_object predictions. The original DETR already outputs the correct classes without post processing.

Thanks for your question, Whether to define the no_object class doesn't determine if the model is end-to-end.  End-to-end object detection means no post-processing like NMS is needed, and the network get the image as input, then output the detection results directly.


Flash of blue body.

Because the body has a blue background that is overlayed by a gray / white content when refreshing the page it shows a flash of blue, similar to a "flash of unstyled content".

Hello,
To be honest I'm not quite sure what you mean, can you please provide a screenshot ?

We're gonna close this ticket for now, but you can reopen it anytime :)

Oh, I missed your request for more info.
Are you familiar with the flash of unstyled content?  https://en.wikipedia.org/wiki/Flash_of_unstyled_content
Your template suffers from it due to the fact that the body has a different background color than the content div.

Ah, I got it now. Ok I will investigate this and come with a solution, thank you for pointing this out !

So did you fix this? I see you closed it  but no commit mentions it?

I've also seen this. Usually when the page has short content, the init_sidebar waits for the css/js files to load, and then init_sidebar adjusts the vertical position of the footer.
Sometimes when the page has long content, the footer can be seen about 10 pixels or more lower than the content body, and then the dark background is again visible.
Ideally, if the footer could be positioned using plain HTML instead of JS would solve this issue as well as my issue #499. Then with only additional CSS issue #498 can be solved..

Well @SamMousa your issue was been solved yes, everything in this milestone that's closed means that was fixed if is not you'll get an explanation instead for sure.
@nkavian both of your issues are in this milestone, if you noticed something else please open another issue to add it here ( this milestone ) :)

Hello @AndreiCN I'm using the django flavor and when I have short content a blue footer appears and there is other scenario that I posted here:  https://stackoverflow.com/questions/46473264/blue-footer-in-gentelella-theme Thanks for the awesome work!


Get 404 error in Firefox console when trying to login as admin

Did you check the FAQ & Troubleshooting section for answers to common questions and issues?

Yes

Describe the issue

What happened? Post any relevant log snippets.
Whenever I try to login on my instance (new install) it loads quickly, then does nothing. Upon opening the console in Firefox, it shows a 404 error and states that "https://my-lemmy-instance.com/api/v3/user/login" is not found. Made a test account just fine, cannot log into it due to needing to approve it with the admin account.

Some cookies are misusing the recommended “SameSite“ attribute 2
Source map error: Error: request failed with status 400
Resource URL: https://lemmy.lat/css/themes/darkly.css
Source Map URL: darkly.css.map
XHRPOST
https://lemmy.lat/api/v3/user/login
[HTTP/2 404 Not Found 84ms]

Diagnostic Information

Run ./deploy.sh -d and paste the output below:

==== Docker Information ====
Detected runtime: docker (Docker version 24.0.2, build cb74dfc)
Detected compose: docker compose (Docker Compose version v2.18.1)
   Runtime state: OK

==== System Information ====
      OS: Linux
  KERNEL: 5.15.0-76-generic (x86_64)
HOSTNAME: OK
   SHELL: bash
  MEMORY:
               total        used        free      shared  buff/cache   available
Mem:            23Gi       914Mi        15Gi        49Mi       7.2Gi        22Gi
Swap:           51Gi          0B        51Gi

DISTRO:
----------------------------
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
----------------------------

==== Lemmy-Easy-Deploy Information ====
Version: 1.2.7

IMAGE                                CREATED          STATUS
caddy:latest                         18 minutes ago   Up 18 minutes
ghcr.io/ubergeek77/lemmy-ui:0.18.1   18 minutes ago   Up 18 minutes
ghcr.io/ubergeek77/lemmy:0.18.1      18 minutes ago   Up 18 minutes
asonix/pictrs:0.4.0-beta.19          18 minutes ago   Up 18 minutes
postgres:15-alpine                   18 minutes ago   Up 18 minutes

Integrity:
    0dcb79d994394bd204f7034ce2bd58838d9d1187262463c95d3abb70a1d6d421  ./deploy.sh
    1e9b0c0988998dcc33cb0fbfdb0e1679229424e724f898b797380adc7d102446  ./templates/Caddy-Dockerfile.template
    c1202e70662dd2228da36a35a0f38ec8fc81bec8964d7315d02e8671a58dd7d7  ./templates/Caddyfile.template
    2537678c7971df36c1ed95f4228d3cfcb15bb4a28a60d939eaf8dd75b5d64a36  ./templates/cloudflare.snip
    c9cb4c5fee12930e17798a02ae1bd12e2dc69e149a394c24511bc9d4e6b776d4  ./templates/compose-email.snip
    c494a610bcb4cd1cfc0a4fe4fb0f6d437b2a84a0ad1625daee240e6dd6f1c910  ./templates/compose-email-volumes.snip
    d9928baea61975bd432e1a4a7439c04468547c9b42e2bb483c6f18d9438665eb  ./templates/docker-compose.yml.template
    1c202b1b6e87c65b2fcda6035c9fe3f8631d76662907ffd38f24b14686e30647  ./templates/lemmy-email.snip
    c834cdce9eaf77f38155b404724fdfe66845575386ee516987452aa715642a6f  ./templates/lemmy.hjson.template

Custom Files:
*** No custom files ***

==== Settings ====
        CLOUDFLARE: No
 CADDY_DISABLE_TLS: false
   CADDY_HTTP_PORT: 80
  CADDY_HTTPS_PORT: 443
 LEMMY_TLS_ENABLED: true
      ENABLE_EMAIL: false
         SMTP_PORT: 25
    ENABLE_POSTFIX: false
POSTGRES_POOL_SIZE: 5

==== Generated Files ====
Deploy Version: 0.18.1;0.18.1

total 36K
drwxrwxr-x 2 1001 1002 4.0K Jul  7 03:30 caddy
-rw-rw-r-- 1 1001 1002   25 Jul  8 00:39 caddy.env
-rw-rw-r-- 1 1001 1002 1.7K Jul  8 00:39 docker-compose.yml
-rw-rw-r-- 1 1001 1002   50 Jul  7 03:30 lemmy.env
-rw-rw-r-- 1 1001 1002  459 Jul  8 00:39 lemmy.hjson
-rw-rw-r-- 1 1001 1002   49 Jul  7 03:30 pictrs.env
-rw-rw-r-- 1 1001 1002   29 Jul  8 00:39 postfix.env
-rw-rw-r-- 1 1001 1002   51 Jul  7 03:30 postgres.env
-rw-rw-r-- 1 1001 1002   14 Jul  8 00:39 version

What happens if you visit this URL:
https://your-instance.com/api/v3/site

If it loads and you see a JSON response, then my routing is working properly, and I may not be able to help you.
If it also shows 404, then I will need to look into why it broke, but I have no idea why it would break so suddenly.

Yep I got a JSON response. Where do I report my issue?

Most likely the UI issue tracker:
https://github.com/LemmyNet/lemmy-ui/issues
However, note that https://my-lemmy-instance.com/api/v3/user/login is a POST-only endpoint. If you try running a GET request to it, it will respond 404.
Are you sure you can't log in using the generated admin password? Be sure to not copy the quotation marks, as they are not part of the password.

Yeah, the password is correct.

This is what shows, it is making a POST request I believe.

Wait hold on, when under the "Response" tab, it says "Cannot find username or email"

I've just deployed a fresh 0.18.1 instance and sadly cannot reproduce this issue, I can log in as normal.
And yes, I just tried what you mention, if I give it an intentionally bad password, the /login endpoint also returns 404 just like this. So your password is simply incorrect.
Note that the default username is lemmy, not admin.

I need sleep oh my god. I am so sorry, I have been typing in admin rather than lemmy. My bad, sorry for the trouble.

No problem!
One more quick note - if you wanted to start over and have your admin user be admin, you can absolutely do that, just set SETUP_ADMIN_USER to admin in config.env
Note this is only used for setup, so only as long as the postgres volume does not exist. If you wanted to clear out everything and change your user to admin, do the following (this will delete your instance and all generated secrets):
# Change SETUP_ADMIN_USER in config.env to whatever you like, such as admin
./deploy.sh --shutdown
docker volume rm lemmy-easy-deploy_postgres_data
rm -rf ./live
./deploy.sh

Have fun using Lemmy! And enjoy a good rest! 😅

Good to know, thanks. And thanks for the prompt replies, this is a great project and you're a great dev!


[SPARK-6544][build] Increment Avro version from 1.7.6 to 1.7.7

Fixes bug causing Kryo serialization to fail with Avro files in between stages.
https://issues.apache.org/jira/browse/AVRO-1476?focusedCommentId=13999249&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13999249

Can one of the admins verify this patch?

We should probably sync this to the underlying Hadoop version. However I know this already doesn't match it and I suspect that bumping a maintenance release won't hurt. If you have a moment would be good to see which version of Hadoop uses this or else what if any incompatible changes may exist. I call it out since I had some issues with older Avro releases in the past.

Here are the 3 issues resolved in 1.7.7 compared to 1.7.6
https://issues.apache.org/jira/browse/AVRO/fixforversion/12326041/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel
AVRO-1632 - is marked as not a problem so there was no code change
AVRO-1439 - looks like a new API so it is new code as opposed to modification of existing code
AVRO-1476 - is the issue causing the GenericRecord problem and the motivation behind the version upgrade
So it looks like a low risk change.
The Hadoop MR client doesn't appear to specify the avro package version so it should be on 1.7.7 since Avro 1.7.7 was released on 07/23/2014.
https://github.com/apache/hadoop/blob/3c9181722b05a9192f5440ea8f3f77231f84eac6/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml

Yeah I think it's pretty safe, LGTM.

Yeah LGTM - I think bumping maintenance releases like this has usually been fine for us with Avro. It may be best not merge this into 1.3 though, since we typically keep dependencies fixed in patch releases.

ok to test

Test build #29303 has started for   PR 5193 at commit 813d4c5.

This patch merges cleanly.

Test build #29303 has finished for   PR 5193 at commit 813d4c5.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29303/
Test PASSed.

@pwendell this is a blocker for any jobs using Avro and Kryo that passes GenericData.Records between stages so I think you should consider merging it in to 1.3 so it will be available in the next patch release. The other change from Avro 1.7.6 to 1.7.7 is just a new API so should be very low risk.

It's a fair point - in some cases we have made minor (patch) release updates in our own patch versions. @srowen what do you think? It can cause some friction for packagers if dependencies change in these patch releases, but this upgrade may be alright.

Okay after reviewing the changes to Avro I've decided to backport this. I looked a bit more and in other cases we have done fairly small dependency updates like this provided the changes in the dependency have been minimal between versions.

Thanks!


frontend: Give Feedback of expired but claimed request (not withdrawable)

A potential griefing attack:
An unfilled request is claimed. As a consequence, the request is assigned to an active claim. Since this claim will not be resolved until the end of finalization period of target chain, the deposit is stuck for the end user. Even worse, if the request is expired (let's say after 10 minutes), nobody can fill the expired request but the active claim needs to be resolved before the deposit can be withdrawn, (let's say 7 days).
The bare minimum we can do in such an unfortunate situation is to let the user know whats going on, as neither got the request filled nor it is withdrawable after expiry.
In the long term, we need to find a solution for that.
Maybe, the user could be able to change the request parameters (increase validity period, increase fees, etc.)

closed in favor of #606


It would be good not to hang the image when setting a border to a color object instead of a border

Interrupt does not work here


Integration and Stress tests for Rate Limiter implemented using DiceDB for Playground Backend

Description
DiceDB Playground is an interactive platform that allows users to experiment with DiceDB commands in a live environment, similar to the Go Playground. Users will be able to submit and execute commands directly through a web interface. The initial implementation will focus on enabling users to search for and trigger commands.
For more details about DiceDB playground you can check here.
As part of this task we need to enhance unit tests, integration tests and stress test Rate Limiter implemented using DiceDB for Playground Backend.
This task would be extension of https://github.com/DiceDB/dice/issues/729
Details

Test the implementation under load to ensure it correctly blocks requests after 1000 requests/minute.
Test with fewer than 1000 requests per minute, ensuring no requests are blocked.
Test with more than 1000 requests per minute, ensuring that requests beyond the limit are blocked and receive a 429 response.
Ensure the rate limit resets correctly after each minute.
Validate the correctness of rate-limiting headers (x-ratelimit-limit, x-ratelimit-remaining, etc.) in responses.
Test normal API /cli behaviour when under the rate limit (e.g., under 1000 requests/minute).

Dependent on https://github.com/DiceDB/dice/issues/729

@lucifercr07 I would like to work on this

hey @lucifercr07 i would too like to work on this

Closing, merged as part of https://github.com/DiceDB/playground-mono/pull/13


This library cannot reconcile with the react-native onnxuruntime official library: unresolved OrtGetApiBase symbol

using libonnxruntime.so from the official react native library:

sherpa-onnx will say cannot resolve "OrtGetApiBase" symbol. I think this is because the official library has versioned API.

make the official library use the libonnxruntime.so downloaded from your release page: crashes when using the official library features to run other onnx models

Is there any way we can reconcile this?

make the official library use the libonnxruntime.so downloaded from your release page: crashes when using the official library features to run other onnx models

Do you re-build sherpa-onnx using linbonnxruntime.so from the official react native library?

No, I didn't do that. How can I do that? I'm new to this, please help.
I just downloaded the libs from your release page.

Please see our documentation
https://k2-fsa.github.io/sherpa/onnx/android/index.html

Thanks, I will try that

If you have troubles, please point me to the URL of libonnxruntime.so and I will build one for you.

This is the jettified onnx lib in my transform cache: https://github.com/l3utterfly/jetified-onnxruntime-android-1.17.0
If you can build one I would greatly appreciate it! Thank you!

@l3utterfly
Please see #603

First, I want to check if it works for you.
If yes, then I think we can add documentation for it and also provide pre-built .so files for it.

First, I want to check if it works for you.

Yes, this works!

Thanks for verifying it.
Do you have a plan to contribute the React Native support for sherpa-onnx or will your implementation be open sourced?

Sure, I am happy to help where I can! I don't have any custom implementation though.. I just followed your documentation...

I'm only using the TTS portion of sherpa, so I basically followed your documentation on building for android/ios, then wrote a React Native bridge which exposes the one function that TTS needs.

I'm only using the TTS portion of sherpa, so I basically followed your documentation on building for android/ios, then wrote a React Native bridge which exposes the one function that TTS needs.

Will your binding code be open sourced?

This is the binding code:
package com.k2fsa.sherpa.onnx

import android.net.Uri
import com.facebook.react.bridge.Arguments
import com.facebook.react.bridge.Promise
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
import com.facebook.react.modules.core.DeviceEventManagerModule
import java.io.File
import java.io.FileOutputStream
import java.io.IOException
import java.io.InputStream
import java.nio.charset.StandardCharsets
import java.util.UUID

class Sherpa(reactContext: ReactApplicationContext?) : ReactContextBaseJavaModule(reactContext) {
    private var listenerCount = 0
    private lateinit var tts: OfflineTts

    override fun getName(): String {
        return "Sherpa"
    }

    // this function is called from C++
    private fun callback(samples: FloatArray) {
        //track.write(samples, 0, samples.size, AudioTrack.WRITE_BLOCKING)
    }

    @ReactMethod
    fun initTts(modelDir: String, modelName: String, modelDataDir: String) {

        var ruleFsts: String? = null
        var lexicon: String? = null

        val config = getOfflineTtsConfig(
            modelDir = modelDir!!, modelName = modelName!!, lexicon = lexicon ?: "",
            dataDir = modelDataDir ?: "",
            ruleFsts = ruleFsts ?: ""
        )!!

        if(::tts.isInitialized)
            tts.free();

        tts = OfflineTts(assetManager = null, config = config)
    }

    @ReactMethod
    fun generate(text: String, voiceId: Int, speed: Float, genDir: String, promise: Promise) {
        Thread {
            try {
                val audio = tts.generateWithCallback(
                    text = text,
                    sid = voiceId,
                    speed = speed,
                    callback = this::callback
                )

                // Create a File object for the directory
                val directory = File(genDir)
                // Check if the directory exists, if not, create it
                if (!directory.exists()) {
                    directory.mkdirs()  // This will create the directory including any necessary but nonexistent parent directories.
                }

                val filename = "$genDir/${UUID.randomUUID()}.wav"
                val ok = audio.samples.size > 0 && audio.save(filename)
                if (ok) {
                    // If saving is successful, resolve the promise
                    promise.resolve(filename)
                } else {
                    // If saving failed, reject the promise
                    promise.reject("ERROR", "Failed to generate or save audio.")
                }
            } catch (e: Exception) {
                // Handle any exceptions by rejecting the promise
                promise.reject("ERROR", "Exception occurred: ${e.message}")
            }
        }.start()
    }
}

Currently it only exposes what I need in my application, but it could be a good starting point. It's not encapsulated in its own library or anything.

I'm not sure how useful this is. I did not create a separate package or anything.


Add a mapping from edges to paths with the dummy nodes

This makes solve_positions also return a Dict which is a mapping from edges in the original graph to paths in the graph with dummy nodes used to solve the layout problem.
The Dict has for keyset edges(g) and it's values are tuples of vectors : x and y coordinates.
I updated the examples with the output we get by plotting these paths, but the simpler examples don't look so great :

Why don't we have a function solve_positions_full that returns all the points, and the full graph?
It has all the nodes of the original_graph at the original index, plus all the dummy nodes and their new edges.
then the current solve_positions would just mask-out the dummy-nodes
I don't fully understand this new structure of paths and what it is for.

Hmm my use case is plotting the output of some maxflow problem.
For each edge in the original graph I want to annotate the plotted edge with the flow passing through it.
If we return the full graph with the new edges, it makes it much harder to do that since I want to annotate a path in the full graph and not each edge individually.
And then, some information is lost : which edge in the full graph corresponds to which edge in the original graph ?
So that's why I went with a mapping from the edges in the original graph to paths in the full graph

Ah, I see.
This makes sense now.
Needs doc strings explaining what paths is.

New docstring looks great.
why is CI failing?
We should have two sets of reference tests.
One set using the coordinates directly and one set using the paths.

Sure,
Is there an allow failure option on nightly ? I'm haven't migrated any of my packages to Gitlab CI yet so I'm not sure but it seems related to the failure on nightly

I have added the plots to the readme.
The first one is a good demonstration of the difference this make.

Sorry, I lost track of this.
I think just need to make CI pass (which might mean disabling Julia nightly)
and then we are good to merge.
Also want to decide what to do with curves.
I am now leaning towards just mentioning it in inhe README.md, so we can merge this sooner, rather than having to go through the effort of regenerating the plots.

Rebasing this should fix the CI

bump, this would be good to have

Rebased it, sorry for the delay, hard to find motivation these days :)

Rebased it, sorry for the delay, hard to find motivation these days :)

Big  Mood.
(that is what the young people say, right?)


Warnings security

Description of changes
I added some warnings when using rest as a settings and additionally in the docker_startup shell script.
closes #347
Discussed with @jeffchuber on discord too.

We'd rather rely on the documentation to let users know as opposed to additional texts into every boot.


Crash webpack 5 at the end of compilation

Hello,
I meet a crash with this error at the end of the compilation:
ERROR in [initial] main.e4e732f00dfe0da35af2.js
Maximum call stack size exceeded

webpack 5.11.0 compiled with 1 error in 38480 ms

Any clue?
I splitted lots of modules into small chunks.
Here is webpack config
COMMON :

module.exports = {
  context: sourcePath,
  mode,
  entry: {
    main: './main.tsx'
  },
  output: {
    path: outPath,
    filename: '[name].[fullhash].js',
    sourceMapFilename: '[name].[fullhash].map',
    chunkFilename: '[id].[fullhash].js',
    publicPath: '/'
  },
  target: 'web',
  resolve: {
    extensions: ['.js', '.ts', '.tsx'],
    mainFields: ['module', 'browser', 'main'],
    alias: {
      app: path.resolve(__dirname, 'src/app/'),
      types: path.resolve(__dirname, 'types/'),
      assets: path.resolve(__dirname, 'src/assets/')
    },
    fallback: {
      fs: false,
      net: false
    }
  },
  module: {
    rules: [
      // .ts, .tsx
      {
        test: /\.tsx?$/,
        use: isProduction
          ? ['ts-loader']
          : [
              {
                loader: 'babel-loader',
                options: {
                  plugins: ['react-hot-loader/babel']
                }
              },
              'ts-loader'
            ]
      },
      // css
      {
        test: /\.css$/,
        use: [
          {
            loader: MiniCssExtractPlugin.loader
          },
          {
            loader: 'css-loader',
            options: {
              fallback: 'style-loader',
              esModule: false,
              modules: {
                localIdentName: '[local]__[fullhash:base64:5]'
              },
              sourceMap: !isProduction,
              importLoaders: 1
            }
          },
          {
            loader: 'postcss-loader',
            options: {
              postcssOptions: {
                ident: 'postcss',
                plugins: [
                  require('postcss-import')({ addDependencyTo: webpack }),
                  require('postcss-url')(),
                  require('postcss-cssnext')(),
                  require('postcss-reporter')(),
                  require('postcss-browser-reporter')({
                    disabled: isProduction
                  })
                ]
              }
            }
          }
        ],
        sideEffects: true,
      },
      // static assets
      { test: /\.html$/, use: 'html-loader' },
      { test: /\.png$/, use: 'url-loader?limit=10000' },
      { test: /\.jpg$/, use: 'file-loader' },
      { test: /\.svg$/, use: 'file-loader' },
      {
        test: /\.(webm|mp4)$/,
        use: {
          loader: 'file-loader',
          options: {
            name: 'videos/[name].[ext]'
          }
        }
      },
      {
        test: /\.(woff|woff2|eot|ttf|otf)$/,
        use: {
          loader: 'url-loader',
          options: {
            name: 'fonts/[name].[ext]',
            limit: 1
          }
        }
      }
    ]
  },
  optimization: {
    splitChunks: {
      cacheGroups: {
        commons: {
          chunks: 'initial',
          minChunks: 2
        },
        vendors: {
          test: /[\\/]node_modules[\\/]/,
          chunks: 'all',
          priority: -10
        }
      }
    },
  },
  plugins: [
    new FixStyleOnlyEntriesPlugin(),
    new CaseSensitivePathsPlugin(),
    new CleanWebpackPlugin(),
    new MiniCssExtractPlugin({
      filename: '[name].[fullhash].css',
      chunkFilename: '[name].[fullhash].css'
    }),
    new HtmlWebpackPlugin(index),
    new RobotstxtPlugin(robotsOptions),
    new SitemapWebpackPlugin({
      base: 'https://' + domain,
      paths,
      options: {
        fileName: 'sitemap.xml',
        lastMod: true,
        changeFreq: 'monthly',
        priority: 0.5
      }
    }),
    new webpack.DefinePlugin(
      configFromEnv
    ),
    /*
    new WorkboxPlugin.GenerateSW({
      clientsClaim: true,
      skipWaiting: true,
      maximumFileSizeToCacheInBytes: 20 * 1024 * 1024
    }),
    */
    new CopyWebpackPlugin({
      patterns: [
        { from: 'assets/pwa/manifest.json', to: 'pwa/manifest.json' },
        { from: 'assets/pwa/**/*', to: '[name].[ext]' },
        { from: 'assets/main.css', to: 'main.css' },
        { from: 'assets/fonts/**.*', to: 'fonts/[name].[ext]' },

      ],
    }),
    new WebpackPwaManifest(manifest),
    new WorkboxPlugin.InjectManifest({
      swSrc: manifest.serviceworker, // './assets/pwa/service_worker.js',
      swDest: 'service-worker.js'
    }),
    new FaviconsWebpackPlugin(
      path.resolve(__dirname, 'src/assets/pwa/favicon-96x96.png')
    )
  ],
  devServer: {
    contentBase: sourcePath,
    hot: true,
    inline: true,
    disableHostCheck: true,
    historyApiFallback: {
      disableDotRule: true
    },
    stats: 'minimal'
  },
  devtool: 'eval-source-map',
};

PROD :
merge(common, {
  mode: 'production',
  externals: {
    "mapbox-gl": "mapboxgl"
  },
  optimization: {
    minimizer: [terser],
    runtimeChunk: 'single',
    minimize: true,
    usedExports: true,
    providedExports: true,
    concatenateModules: true,
    moduleIds: 'deterministic',
    flagIncludedChunks: true,
    mergeDuplicateChunks: true,
    removeEmptyChunks: true,
    chunkIds: 'total-size',
    moduleIds: 'size',
    splitChunks: {
      chunks: 'all',
      maxInitialRequests: 100,
      minSize: 0,
      cacheGroups: {
        common: {
          chunks: 'async'
        },
      'styled-components': {
          name: 'styled-components',
          chunks: 'async',
          filename: 'styled.components.[name].[fullhash].bundle.js',
          test: /[\\/]node_modules[\\/]styled-components[\\/]/
        },
        'react-html-parser': {
          name: 'react-html-parser',
          chunks: 'async',
          filename: 'html.parser.[name].[fullhash].bundle.js',
          test: /[\\/]node_modules[\\/]react-html-parser[\\/]/
        },
        node_modules: {
          name: 'node_modules',
          chunks: 'async',
          enforce: true,
          filename: '[name].[fullhash].bundle.js',
          test: /[\\/]node_modules[\\/](?!(@mapbox|mapbox-gl|styled-components|react-popupbox|dompurify|react-responsive|react-mapbox-gl|react-audio-player|react-html-parser|formik|date-fns|react|react-lottie|lottie-web|mobx-react|mobx-react-router|react-router|react-mobx|mobx|sanitize-html|react-backdrop-filter|@sentry|@sentry\/core|@sentry\/hub|@sentry\/minimal|@sentry\/browser)[\\/])/
        }
      }
    }
  },
  plugins: [
    terser,
    new webpack.DefinePlugin({
      'babel-plugin-styled-components': {
        pure: true
      }
    }),
    new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/),
    /*
    new BundleAnalyzerPlugin({
      analyzerPort: 8886,
    }),
    */
    new LodashModuleReplacementPlugin(),
    // new webpack.optimize.AggressiveMergingPlugin(),
    new CompressionPlugin({
      filename: '[name].[fullhash].gz[query]',
      algorithm: 'gzip',
      test: /\.js$|\.ts$|\.tsx$|\.css$|\.html$/,
      minRatio: 1
    })
  ]
});

Thanks a lot

hmm.. difficult to say. Do you have more details? Try --stats-error-details or --stats-error-stack

--stats-error-stack gives :

EntrypointsOverSizeLimitWarning: entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
  main (6.06 MiB)
      runtime.212175d7c1560dfdd270.js
      @sentry/core.212175d7c1560dfdd270.bundle.js
      @sentry/browser.212175d7c1560dfdd270.bundle.js
      react-dom.212175d7c1560dfdd270.bundle.js
      929.212175d7c1560dfdd270.js
      main.212175d7c1560dfdd270.js

    at compiler.hooks.afterEmit.tap.compilation (/Server/cyberlife/node_modules/webpack/lib/performance/SizeLimitsPlugin.js:139:7)
    at Hook.eval [as callAsync] (eval at create (/Server/cyberlife/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:14:1)
    at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/Server/cyberlife/node_modules/webpack/node_modules/tapable/lib/Hook.js:18:14)
    at asyncLib.forEachLimit.err (/Server/cyberlife/node_modules/webpack/lib/Compiler.js:799:27)
    at /Server/cyberlife/node_modules/neo-async/async.js:2818:7
    at done (/Server/cyberlife/node_modules/neo-async/async.js:3522:9)
    at alreadyWritten (/Server/cyberlife/node_modules/webpack/lib/Compiler.js:645:8)
    at outputFileSystem.readFile (/Server/cyberlife/node_modules/webpack/lib/Compiler.js:730:19)
    at /Server/cyberlife/node_modules/graceful-fs/graceful-fs.js:123:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:53:3)

ERROR in [initial] main.212175d7c1560dfdd270.js
Maximum call stack size exceeded
ChunkRenderError: Maximum call stack size exceeded
    at errorAndCallback (/Server/cyberlife/node_modules/webpack/lib/Compilation.js:3250:26)
    at assetCacheItem.get (/Server/cyberlife/node_modules/webpack/lib/Compilation.js:3334:9)
    at hooks.get.callAsync (/Server/cyberlife/node_modules/webpack/lib/Cache.js:93:5)
    at Hook.eval [as callAsync] (eval at create (/Server/cyberlife/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Cache.get (/Server/cyberlife/node_modules/webpack/lib/Cache.js:75:18)
    at ItemCacheFacade.get (/Server/cyberlife/node_modules/webpack/lib/CacheFacade.js:116:15)
    at asyncLib.forEach (/Server/cyberlife/node_modules/webpack/lib/Compilation.js:3232:22)
    at arrayEach (/Server/cyberlife/node_modules/neo-async/async.js:2405:9)
    at Object.each (/Server/cyberlife/node_modules/neo-async/async.js:2846:9)
    at asyncLib.forEach (/Server/cyberlife/node_modules/webpack/lib/Compilation.js:3221:14)

Another thing :
Until now it was working great with v4. Now, with v5, it ends with this, and when I run the app, it's really slow and bugging everywhere... I did updates for all dependencies, maybe it's not only webpack of course, but that's strange because no error in console or something, it's just... slow !!
A clue why with webpack 5 my chunks are much bigger and slow in runtime? :/

A clue why with webpack 5 my chunks are much bigger and slow in runtime? :/

Please don't mix questions in one issue, otherwise it will get confused and lose thread and you will not get solution
Please provide reproducible test repo, I think you are using incompatibility plugin
Also you ignore our the issue template, so don't expect a quick response, respect time of other developers

I have the same problem

@Gladkiy-Alexey avoid spamming "same", please create reproducible test repo

What is the node version used? I had a similar issue migrating, updated to use, 14.5.0 it worked for me, give a try.

What is the node version used? I had a similar issue migrating, updated to use, 14.5.0 it worked for me, give a try.

I was using the latest. Finally I found the cause, it was due to a plugin (I upgraded mini-css-extract-plugin and it conflicted with another one that was used to fix a problem with extract-text-plugin used before).
It was not directly due to webpack, the real problem is the error was caught and impossible to find where it came from.
Meanwhile, I think you shouldn't catch those errors and let plugins crash.
I found by exploring the source code in node_modules...

I was using the latest. Finally I found the cause, it was due to a plugin (I upgraded mini-css-extract-plugin and it conflicted with another one that was used to fix a problem with extract-text-plugin used before).
It was not directly due to webpack, the real problem is the error was caught and impossible to find where it came from.
Meanwhile, I think you shouldn't catch those errors and let plugins crash.
I found by exploring the source code in node_modules...

Unfortunately, I don't think we can fix it or improve it on our side, because it is Maximum call stack size exceeded

Unfortunately, I don't think we can fix it or improve it on our side, because it is Maximum call stack size exceeded


Use Swift 5.4 for Windows job on GitHub Actions

I've updated the Windows job on GitHub Actions to use a release build of Swift 5.4, and also updated the dependency in Package.swift and Package.resolved.
Additionally, all GitHub Actions triggers were updated to prevent jobs from running twice on pull requests created from branches in the same repositories.

@googlebot I signed it!

@compnerd would you have a moment to have a look?

Ah, I wonder if its the MSVCRT -> CRT renaming that you are referring to.  I think that we could just do something like:
#if os(Windows)
#if canImport(CRT)
import CRT
#else
import MSVCRT
#endif
#endif

As far as I understand, this was fixed in the latest version of SAP, the current locked version fails to build with these kind of errore:
swift-argument-parser\Sources\ArgumentParser\Usage\HelpGenerator.swift:299:13: error: cannot find type 'CONSOLE_SCREEN_BUFFER_INFO' in scope

Would you like me to bring back the SAP update to this PR?

I've confirmed in my fork https://github.com/MaxDesiatov/swift-benchmark/pull/2 that it won't build with the old snapshot currently used in this repository.
I've created a separate PR on this repo for SAP update, just in case. But that will fail too, as it did in my fork. https://github.com/google/swift-benchmark/pull/90
So I think that SAP update and CI update to Swift 5.4 should be included in this same PR for CI to be all green, unless I'm missing something?

It can be made to work - s-a-p uses canImport to import CRT without the fallback for MSVCRT and that changed in the 5.4 release.  I suppose that its not worth the effort to split out the changes, but in general, it is preferable to stage the versions before bumping the release IMO.

it is preferable to stage the versions before bumping the release IMO.

I totally agree, but I'm still a bit confused as to how to land these two changes separately and to keep their respective CI status green at the same time. For now I'm bringing back the update to s-a-p version, but keeping it as a separate commit. That would preserve the separation after merge if you don't squash it. Ready for review and a CI run.

I was thinking that we could inject a CRT module that just does @_exported import MSVCRT.  But again, I'm not too happy with that either.  Sorry for the churn here.  I think that the original idea of just bumping it together is fine.

Thanks! Would it be possible to tag a new version so that other libraries targeting Windows could depend on that instead of the main branch?

@regexident I see you've tagged the previous release, would you be able to tag a new one with this change included?

@MaxDesiatov I did? On mobile right now so can't check, but I doubt it since I'm just a random contributor like you. 😄

Well, this is weird...

@MaxDesiatov I've tagged 0.1.1; I didn't see a reason for any thing bigger in terms of a version bump, the only change was for newer Swift releases on WIndows.

Fantastic, thanks! 0.1.1 works perfectly fine for this 👍

Oh, interesting. I stand corrected then. 😅


Added the scorecard github action and its badge

[X] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Fixes #7548

cc @saswatamcode


Improving FormWrapper - security enhancements

Is your feature request related to a problem? Please describe.
I would like to improve FormWrapper's security features, which don’t fully protect against common security vulnerabilities. Issues such as inadequate CSRF protection, input sanitization, and limited rate limiting expose applications to potential security risks.
Describe the solution you'd like
I’d like FormWrapper to include several key security enhancements:

Advanced CSRF Protection: Implement a double submit cookie pattern, automatic CSRF token rotation, and SameSite cookie attributes to strengthen CSRF defenses. Middleware for CSRF token validation and improved documentation would help ensure secure usage.

Sanitization Pipeline : Integrate a comprehensive input validation pipeline with DOMPurify for sanitization, HTML encoding for special characters, custom sanitization rules, and file upload validation. Additional checks for maximum field length and nested object validation would further prevent XSS and improve data integrity.

Rate Limiting and DOS Protection: Implement IP-based and sliding window rate limiting along with a token bucket algorithm for more robust DOS protection. Support for rate limit headers, configurable blocked IP lists, and automatic IP blocking would improve resource management and security.

Form Encryption: Enable client-side and end-to-end encryption for sensitive fields. Implement secure key management, encrypted field markers, and documentation to protect sensitive data and ensure privacy compliance.

Security Headers and Configuration: Add essential security headers like Content Security Policy (CSP), X-Frame-Options, X-XSS-Protection, Referrer-Policy, Feature-Policy, and HSTS support to enhance browser security and mitigate XSS and iframe risks.

these are somethings we can integrate
Additional context
Integrating these security enhancements would provide a robust defense against common attacks, optimize performance by managing resource usage, and help secure sensitive data more effectively. Using node-cache, DOMPurify, and secure cookie attributes would enable a streamlined and secure implementation. I expect these improvements to enhance security, making FormWrapper a more reliable choice for developers focused on application security.

ok go ahead
@SamparkBhol


Added option list to disable link "Add filter"

I added an option to disable the link "Add filter" if the developer does not wish the option is visible desired model.
Without the option, or to set it to true, the option will appear normally.
config.model 'About' do
  list do
    field :id
    field :title
  end
end

or
config.model 'About' do
  add_filter true
  list do
    field :id
    field :title
  end
end

But, if set false, will not be show in the model list .
config.model 'About' do
  add_filter false
  list do
    field :id
    field :title
  end
end

This can be achieved by marking all fields as filterable false.

The problem is when the list of attributes and models becomes very large . I know this methods , but that the PR would be a facilitator for all attributes and not a filter for them.


[Dashboard] [Dataset] Favorite dataset

Ability to favorite a dataset. We can re-purpose the ❤  functionality that we already have.

[ ] As the user mouses over a dataset, a grayed out star symbol is shown. Users can click on the star to favorite that dataset.
[ ] Already favorited datasets display a yellow star.
[ ] Favorited datasets are always displayed at the top of the list.

Waiting for Dataset card acceptance to check that everything works.


[7.x] Add 'morphWithCount' method

The morphWith method was added by PR #28647. It allows you to eager load relations on morphed models.
This PR adds a morphWithCount method that allows you to eager load relationship counts on morphed models. It uses the same syntax as the morphWith method.
An example:
Comment::query()
    ->with(['commentable' => function (MorphTo $morphTo) {
        $morphTo->morphWithCount([Post::class => ['likes']]);
    }])
    ->get();

Would you mind sending in a documentation PR for this? Thanks.


Exclude common words

Hello,
Is there any way to exclude common terms in a language, such as "a", "in", "the" and so on?
Thank you very much for your support.
Best regards,
Rogerio

@rogeriobiondi the easiest way to do this if you are using kibana and elasticsearch is to configure analyzers in elasticsearch. You can read more about it here or here.


Initial setup of testing framework

Rationale: current https://github.com/libero/prc doesn't cover End to End (E2E) test cases which should validate the web app integrity from end user perspective, which results in a risk to not detect bugs in web app, & would require manual testing as compensation (which at the end would impact the team velocity)
Task breakdown:

Implement Page object model (PoM)
impact of not having: non linear maintenance effort to support automated tests & low readability of automated test code
benefit: ensure re-usability of page element locators & results in linear maintenance effort in case of changes in app under test
example: https://github.com/amiya-pattnaik/webdriverIO-with-cucumberBDD/blob/master/test/pageobjects/yahoo-search.page.js

Add webdriver.io as web browser connector
impact of not having: vanilla selenium-webdriver requires additional effort to implement reporting, management layer for different browser support & selenium wrapper functions which require
benefit: webdriver.io is wrapper for selenium-webdriver, which minimizes boilerplate for page element location, includes ready to use reporting plugins (Slack, Jenkins) & utility services (browserstack), includes built in capability to rerun flaky tests.
example: https://webdriver.io/

driver management (support chrome, firefox or browserstack remote driver)
benefit: ensure target browser change via config parameters, which enables cross-browser testing later on

Add implicit screenshot & action evidence reporting
benefit: minimize effort for evidence collection in test step implementation

CC: Hi @BlueReZZ , please, reply if you have any questions regarding above mentioned test framework features, so far we believe this is the way to go in order to minimize effort & increase velocity on PRC Article Hosting project.

This looks like a good direction for the end to end tests. The prc repo will have similar at some point but the team have not prioritised it yet. The Page Object Model is something we've used in the past, as is webdriver so a good choice for us. One of our developers is moving from an older project to The Hive in the next few weeks and will be working on end to end tests for that. His previous work is in https://github.com/libero/reviewer-client/tree/master/tests and you can see that the tooling and approach is similar to this proposal.


Clear the terminal after each roll

After the dice gets rolled clear the terminal such that the initial grid gets cleared to make the new grid look better.

Hi, I would like to try to work on this issue, please assign this to me.

Assigned

please come up with a solution that works for all operating system

Sure, will try my best


[docs] Fix signup link for TileDB Community Slack

Issue and/or context: For #1362

Codecov Report
Patch coverage has no change and project coverage change: +25.89 :tada:

Comparison is base (c799b01) 65.11% compared to head (7194861) 91.01%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1369       +/-   ##
===========================================
+ Coverage   65.11%   91.01%   +25.89%
===========================================
  Files          98       30       -68
  Lines        7798     2416     -5382
===========================================
- Hits         5078     2199     -2879
+ Misses       2720      217     -2503

Flag
Coverage Δ

python
91.01% <ø> (ø)

r
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 68 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


trying to add multiple sub menu items to top Nav

Which exact Umbraco version are you using? For example: 8.7.0 - don't just write v8 or v9
11.1.0
Bug summary
I am trying to create multiple sub menu items under main navigation and unable to see the items after they are added on the screen.
Specifics
No response
Steps to reproduce
this can be reproduced by trying to add multi level sub menu items under main nav
Expected result / actual result
No response

You need to write the foreach look for each set of child items.
They can be found in Children
See #70 for more about this discussion and example code


Master to Live

Docs Build status updates of commit 9924d8b:
:clock10: Full build: the average full build time is 13 min(s) 58 sec(s), this is based on the last 412 build(s) for this repository.

Docs Build status updates of commit 9924d8b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/advanced-threat-protection/New-AntiPhishPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/advanced-threat-protection/Set-AntiPhishPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/organization/Set-OrganizationConfig.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/users-and-groups/Get-UnifiedGroupLinks.md
:white_check_mark:Succeeded
View (exchange-ps)

sharepoint/sharepoint-ps/sharepoint-online/Get-SPOGeoMoveCompatibilityStatus.md
:white_check_mark:Succeeded
View (sharepoint-ps)

sharepoint/sharepoint-ps/sharepoint-pnp/New-PnPWeb.md
:white_check_mark:Succeeded
View (sharepoint-ps)

skype/skype-ps/skype/Move-CsRgsConfiguration.md
:white_check_mark:Succeeded
View (skype-ps)

skype/skype-ps/skype/Set-CsMediaConfiguration.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 63d2758:
:clock10: Full build: the average full build time is 13 min(s) 58 sec(s), this is based on the last 415 build(s) for this repository.

Docs Build status updates of commit 63d2758:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/advanced-threat-protection/New-AntiPhishPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/advanced-threat-protection/Set-AntiPhishPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/organization/Set-OrganizationConfig.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/users-and-groups/Get-UnifiedGroupLinks.md
:white_check_mark:Succeeded
View (exchange-ps)

sharepoint/sharepoint-ps/sharepoint-online/Get-SPOGeoMoveCompatibilityStatus.md
:white_check_mark:Succeeded
View (sharepoint-ps)

sharepoint/sharepoint-ps/sharepoint-pnp/New-PnPWeb.md
:white_check_mark:Succeeded
View (sharepoint-ps)

skype/skype-ps/skype/Move-CsRgsConfiguration.md
:white_check_mark:Succeeded
View (skype-ps)

skype/skype-ps/skype/Set-CsMediaConfiguration.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Simplify fairing chapter

Fairing chapter has two notebooks. Would rather focus on important one and simplify instructions so that its easy to follow
https://eksworkshop.com/kubeflow/fairing/

@dalbhanj is there a link to the code?


sst: generate types without needing to be authenticated

We want to run our typechecker in CI but it needs the autogenerated types in .sst. We can only generate these when running sst dev or sst deploy which requires being authenticated with our provider (AWS). It would be nice to be able to generate the types without needing that.

+1. We ran into something similar and ending up committing our SST types for now.

Bump it

It would be nice. We have a monorepo setup where sst generates 50+ type files which cluttered up our PRs. So we have only committed the root sst type file and then a script runs in the CI environment which copies this root sst type file into all folders that have a package.json file.
This way we have the root type (once) for review and it will be copied for typechecking by the CI. Locally sst just generates all the type files but they are gitignored.

I don't think we can generate the types without building your app. I'll need to double check.


Add Nong Song:

Song Name
Glory (Slowed)
Artist Name
Ogryzek
Source
Youtube
Youtube Link
https://www.youtube.com/watch?v=TwRTUTImFBQ
Direct File Link
No response
Song ID
714579
Start Offset [ms]
No response
Extra
No response

accept

Accept
El mié., 7 de agosto de 2024 1:40 p. m., auto-nong[bot] <
@.***> escribió:

Thanks for contributing!
Someone will soon review your request.
Here is the JSON result for the reviewer:
{
"name": "Glory (Slowed)",
"artist": "Ogryzek",
"source": "youtube",
"startOffset": 0,
"yt-id": "TwRTUTImFBQ",
"songs": [
714579
]
}
To accept the request, a reviewer needs to write "accept".
—
Reply to this email directly, view it on GitHub
https://github.com/FlafyDev/auto-nong-indexes/issues/2524#issuecomment-2274107776,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BKMLKGI256C5V27L75MGM2LZQJS2HAVCNFSM6AAAAABME66VNCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZUGEYDONZXGY
.
You are receiving this because you authored the thread.Message ID:
@.***>


Segmentation fault when building engine

When I tried to build llama engine usingtrtllm-build --checkpoint_dir ./tllm_checkpoint_8gpu_tp8/ --output_dir ./output_engine_1 --max_num_tokens 4096 --max_input_len 131072 --max_seq_len 131082 --gemm_plugin auto, the error information occurs:
[08/06/2024-11:56:31] [TRT] [I] Global timing cache in use. Profiling results in this builder pass will be stored.
[08/06/2024-11:56:35] [TRT] [I] [GraphReduction] The approximate region cut reduction algorithm is called.
[08/06/2024-11:56:35] [TRT] [I] Detected 15 inputs and 1 output network tensors.
[n101-006-033:3841514] *** Process received signal ***
[n101-006-033:3841514] Signal: Segmentation fault (11)
[n101-006-033:3841514] Signal code:  (128)
[n101-006-033:3841514] Failing at address: (nil)
[n101-006-033:3841514] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fce4699b420]
[n101-006-033:3841514] [ 1] /root/miniconda3/envs/torch/bin/../lib/libstdc++.so.6(_ZNSs6assignERKSs+0x9d)[0x7fcda3a429cd]
[n101-006-033:3841514] [ 2] /mnt/llm/TensorRT-LLM/tensorrt_llm/libs/libtensorrt_llm_nvrtc_wrapper.so(tllmXqaJitCreateAndCompileProgram+0x4a2)[0x7fcb29164ef2]
[n101-006-033:3841514] [ 3] /mnt/llm/TensorRT-LLM/tensorrt_llm/libs/libtensorrt_llm.so(_ZNK12tensorrt_llm7kernels3jit13CompileEngine7compileEv+0xb1)[0x7fcb2dbcf941]
[n101-006-033:3841514] [ 4] /mnt/llm/TensorRT-LLM/tensorrt_llm/libs/libtensorrt_llm.so(_ZN12tensorrt_llm7kernels17DecoderXQAImplJIT25prepareForActualXQAParamsERKNS0_9XQAParamsE+0x15a)[0x7fcb2dbd0daa]
[n101-006-033:3841514] [ 5] /mnt/llm/TensorRT-LLM/tensorrt_llm/libs/libtensorrt_llm.so(_ZN12tensorrt_llm7kernels17DecoderXQAImplJIT7prepareERKNS0_9XQAParamsE+0x60)[0x7fcb2dbd1040]
[n101-006-033:3841514] [ 6] /mnt/llm/TensorRT-LLM/tensorrt_llm/libs/libnvinfer_plugin_tensorrt_llm.so(+0xd5b7f)[0x7fcae0046b7f]
[n101-006-033:3841514] [ 7] /mnt/llm/TensorRT-LLM/tensorrt_llm/libs/libnvinfer_plugin_tensorrt_llm.so(+0xf5d26)[0x7fcae0066d26]
[n101-006-033:3841514] [ 8] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xe4f930)[0x7fcd8fe67930]
[n101-006-033:3841514] [ 9] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xe5026c)[0x7fcd8fe6826c]
[n101-006-033:3841514] [10] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xb3183f)[0x7fcd8fb4983f]
[n101-006-033:3841514] [11] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xb3dcdf)[0x7fcd8fb55cdf]
[n101-006-033:3841514] [12] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xb44c5c)[0x7fcd8fb5cc5c]
[n101-006-033:3841514] [13] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xb46e05)[0x7fcd8fb5ee05]
[n101-006-033:3841514] [14] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xaaa78c)[0x7fcd8fac278c]
[n101-006-033:3841514] [15] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xaafa1a)[0x7fcd8fac7a1a]
[n101-006-033:3841514] [16] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_libs/libnvinfer.so.10(+0xab0465)[0x7fcd8fac8465]
[n101-006-033:3841514] [17] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_bindings/tensorrt.so(+0xa84d8)[0x7fcc56d594d8]
[n101-006-033:3841514] [18] /root/miniconda3/envs/torch/lib/python3.10/site-packages/tensorrt_bindings/tensorrt.so(+0x459d3)[0x7fcc56cf69d3]
[n101-006-033:3841514] [19] /root/miniconda3/envs/torch/bin/python3[0x4fdc87]
[n101-006-033:3841514] [20] /root/miniconda3/envs/torch/bin/python3(_PyObject_MakeTpCall+0x25b)[0x4f741b]
[n101-006-033:3841514] [21] /root/miniconda3/envs/torch/bin/python3[0x509cbf]
[n101-006-033:3841514] [22] /root/miniconda3/envs/torch/bin/python3(_PyEval_EvalFrameDefault+0x4b26)[0x4f2c16]
[n101-006-033:3841514] [23] /root/miniconda3/envs/torch/bin/python3(_PyFunction_Vectorcall+0x6f)[0x4fe0cf]
[n101-006-033:3841514] [24] /root/miniconda3/envs/torch/bin/python3(_PyEval_EvalFrameDefault+0x2b79)[0x4f0c69]
[n101-006-033:3841514] [25] /root/miniconda3/envs/torch/bin/python3(_PyFunction_Vectorcall+0x6f)[0x4fe0cf]
[n101-006-033:3841514] [26] /root/miniconda3/envs/torch/bin/python3(_PyEval_EvalFrameDefault+0x731)[0x4ee821]
[n101-006-033:3841514] [27] /root/miniconda3/envs/torch/bin/python3(_PyFunction_Vectorcall+0x6f)[0x4fe0cf]
[n101-006-033:3841514] [28] /root/miniconda3/envs/torch/bin/python3(_PyEval_EvalFrameDefault+0x31f)[0x4ee40f]
[n101-006-033:3841514] [29] /root/miniconda3/envs/torch/bin/python3(_PyFunction_Vectorcall+0x6f)[0x4fe0cf]
[n101-006-033:3841514] *** End of error message ***
Segmentation fault (core dumped)

Can any one solve this problem?
Environment:
Cuda 12.5
tensorrt 10.0.1
tensorrt_llm 0.11.0
tensorrt_bindings 9.0.1.post11.dev4
tensorrt-cu12 10.2.0.post1
tensorrt-cu12-bindings 10.2.0.post1
tensorrt-cu12-libs 10.2.0.post1
tensorrt-libs 9.0.1.post11.dev4

@Kefeng-Duan i encountered the same error. I'm using the llama2-70b model with 4 GPUs, and the command is in examples/llama/README.md to build fp16 model. Do you have any suggestions on how to resolve this?thanks

@zhangts20  which type of GPU?

@zhangts20 which type of GPU?

GPU: 4xA100
tensorrt_llm: v0.11.0
tensorrt: 10.1.0
cuda: 12.1
driver: 525.60.13

Hi, @kevinsouthByteDance could you help to provide more information? are you working on 8 GPUs? what's the model config? and what's the precision?

Hello @Kefeng-Duan @zhangts20 , i have found that this issue is cause by max_position_ids in my model config. When I change the max_position_ids to a larger value, i could build successfully.

Hi, @kevinsouthByteDance could you help to provide more information? are you working on 8 GPUs? what's the model config? and what's the precision?

Hello @Kefeng-Duan @zhangts20 , i have found that this issue is cause by max_position_ids in my model config. When I change the max_position_ids to a larger value, i could build successfully.

I couldn't find the variable max_position_ids in my config.json, do you mean max_position_embeddings?  And what would be an appropriate value to set for it?

Hi, @kevinsouthByteDance could you help to provide more information? are you working on 8 GPUs? what's the model config? and what's the precision?

Hello @Kefeng-Duan @zhangts20 , i have found that this issue is cause by max_position_ids in my model config. When I change the max_position_ids to a larger value, i could build successfully.

I couldn't find the variable max_position_ids in my config.json, do you mean max_position_embeddings? And what would be an appropriate value to set for it? @kevinsouthByteDance

Yes, it is the value of max_position_embeddings, it should be greater than input length + output length. For example, if you wanna test the model with input length: 8192 and output length: 50, then max_position_embeddings should be greater than 8192 + 50 = 8242.

@kevinsouthByteDance sorry for delay replay, how can i set this value when exporting engine, this is a fixed value from config.json. In TensorRT-LLM, --n-positions can change the value of max_position_embeddings only when model_dir is none https://github.com/NVIDIA/TensorRT-LLM/blob/v0.11.0/examples/llama/convert_checkpoint.py#L294

@kevinsouthByteDance sorry for delay replay, how can i set this value when exporting engine, this is a fixed value from config.json. In TensorRT-LLM, --n-positions can change the value of max_position_embeddings only when model_dir is none https://github.com/NVIDIA/TensorRT-LLM/blob/v0.11.0/examples/llama/convert_checkpoint.py#L294

I change this max_position_embeddings in the config file of the huggingface model. Actually there is multiple steps before you build engine, first you should convert the model from a huggingface model(or your own model) to a trtllm checkpoint(you will call convert_checkpoint.py), second you will call trtllm-build to build the engine. And I change the config file before I call the convert_checkpoint.py.

Can we manually change it? I understand it is determined during training. @kevinsouthByteDance @Kefeng-Duan Do you have any other solutions in mind?

@zhangts20  sorry i am not sure i have fully understood 'manually', do you mean what kevinsouthByteDance has done?

@zhangts20 sorry i am not sure i have fully understood 'manually', do you mean what kevinsouthByteDance has done?

Thanks for your reply, that is, how can we solve this problem without modifying the value of max_position_embeddings  in config.json? @Kefeng-Duan

i guess no


fix(withBase): update with pathname instead of path

resolves #878

Can you please add a test as well? 🙏🏼

@pi0 The issue was only happening with toNodeHandler and most probably tied to how the pathname property is set in NodeReqURLProxy. But I put the fix in the common code. So, I am a bit confuse about how I should test it.

@pi0 Tried adding an integration test. Please check.


Update LOCALIZATION_CONTRIBUTORS

Changes
Update LOCALIZATION_CONTRIBUTORS

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I joined GitLocalize on the same github Id.