Flagging enhancement (or, I can't flag dupes, because I can't make them)

Thank you for adding the ability to flag duplicate/deprecated torrents.
There's a bit of a problem though:  I've been remaking all of my old torrents to remove nyaa.se and add new trackers, but I still can't upload them, because the old torrents are there and the system blocks me.  But I can't really flag the old torrents as "duplicates," because they aren't yet.
I could ask to "claim" my old torrents, but as I can't even edit current torrents once uploaded, that's useless for now anyway.  We wouldn't want to just let people edit trackers willy-nilly, that could lead to all sorts of unpleasantness.
Perhaps a further flagging enhancement, like the ability to type in a note on the flag?  Then I could flag as dupe and say "Newer upload pending" or something?

If you want to upload all your torrents again, you can just ask on irc that we remove them all for now or report the ones as duplicate with your account (we will see your username).
I will look into that (if no one else have done it) in 2 days for adding the torrents management tool for users. Like editing your own torrents and editing the trackers of torrents in Bulk.

Okay, I've flagged a whole bunch of them, thanks!

So, uh... is this not gonna happen or something?

Knock knock.

@toynbeeidea Were you the one who reported some "PPP-RAW" torrents?

Should be fixed with the torrent edit api


Fetch sometimes catch a JSON parse error, sometimes it's not

I'm working on a React Native application, which fetches an API response from a server. Sometimes, it returns success, but sometimes it catches an error.
The following code is my fetch function:
export default async (url, body = null, method = 'GET') => {
    let config = {
        method,
    };
    return await fetch(url, config).then((response) => {
        if (!response.ok) {
            throw Error(response.statusText);
        }
        return response.json();
    }).catch(error => {
        console.warn(error); // sometimes it was catched in here
    });
};

It was returning various error messages, e.g:

[SyntaxError: JSON Parse error: "\u302" is not a valid unicode escape]
[SyntaxError: JSON Parse error: Invalid escape character
[SyntaxError: JSON Parse error: "\u740" is not a valid unicode escape]
Unexpected end of json input
[SyntaxError: JSON Parse error: Unexpected character '}']
etc.

Here's a sample of JSON output: https://pastebin.com/vduDQJ8g
I'm already checked my API response through the browser and it didn't find anything wrong with the response. Is there any bug on fetch function so the fetched response is not complete?

Unicode escape sequences in JSON are defined as \u with four hexadecimal digits following.
One way to debug this is to print the value of response.text() rather than passing it through the JSON parser.
The mix of then and async/await can be simplified to this:
export default async (url, body = null, method = 'GET') => {
  let config = {
    method,
  };
  try {
    const response = await fetch(url, config);
    if (!response.ok) {
      throw new Error(response.statusText);
    }
    return await response.json();
  } catch (error)
    console.warn(error);
    throw error;
  }
};

@dgraham I've already debugged and it seems returning an incomplete JSON like this: https://pastebin.com/tE4iYfbc, so the JSON parse was error. Is it a problem on mixing of async/await & fetch?

Is it a problem on mixing of async/await & fetch?

The result of fetch, like any then-able result, is compatible with async/await. The reason @dgraham suggested a slightly different approach was to eliminate the await..then construct that you had in your example, which made your code difficult to read.


Image display in documentation

The logo is not displaying correctly in the readthedocs.io docs.

This page https://spacepy-x.readthedocs.io/en/latest/


[UX2.0] Add support for cli add-on feature profile and config parcel

Description
Add support for cli add-on feature profile and config parcel

Validated in dev-uxmt.


pearson_correlation too slow

Expected behavior
The pearson_correlation operation should execute fast as it uses simple, linear maths.
Actual behavior
It is way too slow. Computing of average size lat/lon images of a few megapixels can easily take minutes.
Steps to reproduce the problem
Run use case 9.
Specifications
0.8.0rc7.dev2

Once we've successfully optimized the operation, e.g. using Numba, we should document this in a dedicated section of the developer guide.

See also #201

Oops, I should have added a comment rather than createing a new issue.

It's fine, more issues to close :) I think I would add it to 'Operation development guidelines' as 'Optimization' or something.

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   124        81          130      1.6      0.0      for lai in range(0, len(lat)):
   125      6480        20297      3.1      0.0          for loi in range(0, len(lon)):
   126      6400     38844444   6069.4     17.1              x = array_x.isel(lat=lai, lon=loi).values
   127      6400    186450590  29132.9     82.1              y = array_y.isel(lat=lai, lon=loi).values
   128      6400      1621394    253.3      0.7              corr_coef[lai, loi], p_value[lai, loi] = pearsonr(x, y)

It's slow not due to the calculation, but rather because of data access across the dask array distributed between files.
forcing explicit reading of the whole selected array into memory:
array_y.load()
array_x.load()

proves the point:
Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   127        81          107      1.3      0.0      for lai in range(0, len(lat)):
   128      6480        14817      2.3      0.1          for loi in range(0, len(lon)):
   129      6400      4614680    721.0     44.9              x = array_x.isel(lat=lai, lon=loi).values
   130      6400      4453011    695.8     43.4              y = array_y.isel(lat=lai, lon=loi).values
   131      6400      1119590    174.9     10.9              corr_coef[lai, loi], p_value[lai, loi] = pearsonr(x, y)

Naively using Numba won't solve the performance issue, rather, it should be explored if the calculation can be vectorized across the array to minimize i/o and leverage parallel processing capabilities of Dask.

Managed to reduce running time of Pearson correlation by a few orders of magnitude:
Total time: 0.773345 s

!

Can we close this now? I mean, is it as fast as can be ;)

I think we should close issues only when relevant code ends up on master. So, once I'm finished with the improvements to correlation, I'll make a PR and denote which Issues would be closed by that PR!


This library is not working properly with live/test modes

I have implemented this library with React Native 0.68.1.
But it's not taking any payment method in either production or development.
always return error
result:::: {"action": "Cancel", "data": {}}
result:::: {"action": "Error", "data": {"paymentMethodType": 0, "transactionId": null}}
Even not return full description of error case
Can anyone help?

which platform are you testing Android or IOS

Thanks for your response
I am trying with both platforms (iOS & Android)
and I am using with
"react-native": "0.68.1"
"ct-react-native-datatrans": "^1.5.1"

Hi SabashChinTam, same issue here can you please help?


Interprocess v1

🫡 Hi everyone, these are some of the planned things that will happen in the Interprocess v1 release!
I'll update this issue on any additions/changes for the release!
Feel free to suggest anything pertinent!! 🤗
Features

[ ] mainSlice - a function that creates an ipc slice for the main process.

it replaces the need of createIpcSlice and combineIpcs usage!

   import { mainSlice, createInterprocess } from 'interprocess'

   // The slice can be moved to another file or folder for better organization
   const pingSlice = mainSlice(async (_, data: string) => { ... })

   export const { ipcMain, ipcRenderer, exposeApiToGlobalWindow } = createInterprocess({
     main: {
       pingSlice,
     }
  })

[ ] rendererSlice - a function that creates an ipc slice for the renderer process.

same as mainSlice but for renderer

Docs

[ ] improve the docs to give more clarity on how to use interprocess in the best way, mainly to extract the best use of ipcs without the need to create dumb ones to only get the types! This has happened because there are no examples of dynamic import in ipcs and this has generated conflicts and misuse of them.

Breaking changes

[ ] combineIpcs (it will be removed on v1)
[ ] createIpcSlice (it will be removed on v1)

Just updated our interprocess to latest as I was trying to figure out a memory leak around invocations, which the latest version does fix in createMainInvokers.ts:
if (!store.main.availableRendererIpcChannels.has(ipcChannel)) {
       return reject(`No handler registered for '${ipcChannel}'`)
}

I wish there was a silent way to reject as we have tons of IPC calls and it'd be a huge pain to wrap try/catch around them. It definitely shouldn't proceed with the subsequent code in this situation, but we'll encounter the error situation at times if our electron version and frontend doesn't match up (as in the FE may be outdated and doesn't register a handler until a later version). We assumed that we can invoke and it will be a technical no-op in this situation since it has no where to send to.

Another feature suggestion is invoker / handler middleware. I have a custom class that wraps the handlers:
  /**
   * This wraps the IPC handlers with another method that will auto-cleanup the handler if it should be
   * re-created again
   */
  private wrapHandlers() {
    this.ipc['handle'] = Object.keys(this.ipc['handle']).reduce((handlers, handlerName) => {
      const handlerFn = this.ipc['handle'][handlerName];

      handlers[handlerName] = (cb: HandlerCallback) => {
        const checkSenderCallback = this.checkSenderOriginMiddleware(handlerName, cb);
        this.cleanupHandler(handlerName);
        return handlerFn(checkSenderCallback);
      };

      return handlers;
    }, {});
  }

The reason for this is because of security - we apply an origin check for any IPC calls from the FE to electron before we proceed with execution. If interprocess had some means to use middleware, this kind of hack wouldn't be necessary.

here's what I did for the invokers
  /**
   * Wraps a try / catch around the invokers to ignore errors around the corresponding handler not
   * being registered (this could be due to an older FE version) when the invoker is called.
   */
  private wrapInvokers() {
    this.ipc['invoke'] = Object.keys(this.ipc['invoke']).reduce((invokers, invokeName) => {
      const invokeFn = this.ipc['invoke'][invokeName];

      invokers[invokeName] = async (window: BrowserWindow, ...args: any[]) => {
        try {
          return await invokeFn(window, ...args);
        } catch (e) {
          // It's ok if we have no handler registered - the FE might be an older version
          // and doesn't register the handler so we ignore this error
          if (typeof e === 'string' && !e.includes('No handler registered for')) {
            this.log.debug(`Error invoking ${invokeName}: ${e}`);
          } else if (e instanceof Error) {
            this.log.error(`Error invoking ${invokeName}: ${e.message}`);
          }
        }
      };

      return invokers;
    }, {});
  }

Hi @theogravity

One note: You can't use interprocess via npm package install if you use sandbox: true for the BrowserWindow prefs because the sandbox won't let you use import node modules in the renderer process since:
A sandboxed renderer won't have a Node.js environment initialized.

To get around this, I had to copy the src files directly to my project in a vendor directory.
Other than that, had really great success with interprocess on our electron app for https://switchboard.app

The docs mention the need of sandbox: false, but I'm thinking what's the best approach in the cases where sandbox: true are not negotiable. Maybe we just have to mention in the docs how to make it work like you did? Any ideas?
About the Switchboard, it really looks awesome, congrats!

Just updated our interprocess to latest as I was trying to figure out a memory leak around invocations, which the latest version does fix in createMainInvokers.ts:
if (!store.main.availableRendererIpcChannels.has(ipcChannel)) {
       return reject(`No handler registered for '${ipcChannel}'`)
}

I wish there was a silent way to reject as we have tons of IPC calls and it'd be a huge pain to wrap try/catch around them. It definitely shouldn't proceed with the subsequent code in this situation, but we'll encounter the error situation at times if our electron version and frontend doesn't match up (as in the FE may be outdated and doesn't register a handler until a later version). We assumed that we can invoke and it will be a technical no-op in this situation since it has no where to send to.
The "fix" I'm doing is something akin to wrapping the invokers like we do handlers (see next comment) by wrapping try / catch around it.

About the rejection, I believe this must be the default behavior, but maybe we should have unsafe handle/invoke methods to not throw? (good to use-as-needed) like zod does in safeParse  (I really don't like naming something like zod calling something as safe and not throwing, unsafe is less confusing, right? What do you prefer?)

Another feature suggestion is invoker / handler middleware. I have a custom class that wraps the handlers:
  /**
   * This wraps the IPC handlers with another method that will auto-cleanup the handler if it should be
   * re-created again
   */
  private wrapHandlers() {
    this.ipc['handle'] = Object.keys(this.ipc['handle']).reduce((handlers, handlerName) => {
      const handlerFn = this.ipc['handle'][handlerName];

      handlers[handlerName] = (cb: HandlerCallback) => {
        const checkSenderCallback = this.checkSenderOriginMiddleware(handlerName, cb);
        this.cleanupHandler(handlerName);
        return handlerFn(checkSenderCallback);
      };

      return handlers;
    }, {});
  }

The reason for this is because of security - we apply an origin check for any IPC calls from the FE to electron before we proceed with execution. If interprocess had some means to use middleware, this kind of hack wouldn't be necessary.

I really like the idea of having middlewares/hooks support on interprocess! 💜
I'll be thinking of a good way to make this happen from a DX perspective!
By the way, thank you very much for the feedback and sponsorship! 💜

I don't really know what would be good DX for the sandbox: true case, but maybe you can have an npx command that would just copy the src files to a directory of the user's choosing?
Regarding the safe proposition, maybe it can just be a config option when calling createInterprocess. When enabled, it'd empty-resolve (not like it matters since there's nothing to handle it anyways) on an invoke without a handler.

The IPC base class we use to create categories of IPCs might help give you ideas:

interface IpcHandlers {
  remove: Record<string, () => void>;
}

type HandlerCallback = (event: IpcMainInvokeEvent, params: any) => Promise<any>;

export interface IpcHandlersParams<T extends IpcHandlers> {
  logPrefix: string;
  ipc: T;
}

/**
 * All IPC handlers extend this class.
 */
export abstract class BaseIpcHandlers<T extends IpcHandlers> {
  /**
   * Set of registered channel names
   */
  private registeredHandlers: Set<string>;
  /**
   * The IPC instance (*ipcMain) from interprocess
   * @protected
   */
  protected ipc: T;

  log: LogLayer;

  protected constructor(params: IpcHandlersParams<T>) {
    this.registeredHandlers = new Set();
    this.log = getMainLogger().withPrefix(`[${params.logPrefix}IPC]`);
    this.ipc = params.ipc;
    this.wrapHandlers();
    this.wrapInvokers();
  }

  abstract initGlobalHandlers(): Promise<void>;

  /**
   * This wraps the IPC handlers with another method that will auto-cleanup the handler if it should be
   * re-created again
   */
  private wrapHandlers() {
    this.ipc['handle'] = Object.keys(this.ipc['handle']).reduce((handlers, handlerName) => {
      const handlerFn = this.ipc['handle'][handlerName];

      handlers[handlerName] = (cb: HandlerCallback) => {
        const checkSenderCallback = this.checkSenderOriginMiddleware(handlerName, cb);
        this.cleanupHandler(handlerName);
        return handlerFn(checkSenderCallback);
      };

      return handlers;
    }, {});
  }

  /**
   * Wraps a try / catch around the invokers to ignore errors around the corresponding handler not
   * being registered (this could be due to an older FE version) when the invoker is called.
   */
  private wrapInvokers() {
    this.ipc['invoke'] = Object.keys(this.ipc['invoke']).reduce((invokers, invokeName) => {
      const invokeFn = this.ipc['invoke'][invokeName];

      invokers[invokeName] = async (window: BrowserWindow, ...args: any[]) => {
        try {
          return await invokeFn(window, ...args);
        } catch (e) {
          // It's ok if we have no handler registered - the FE might be an older version
          // and doesn't register the handler so we ignore this error
          if (typeof e === 'string' && !e.includes('No handler registered for')) {
            this.log.debug(`Error invoking ${invokeName}: ${e}`);
          } else if (e instanceof Error) {
            this.log.error(`Error invoking ${invokeName}: ${e.message}`);
          }
        }
      };

      return invokers;
    }, {});
  }

  /**
   * Takes in the original IPC handler function and wraps it with an origin check before calling
   * the original IPC handler function
   */
  private checkSenderOriginMiddleware(handlerName: string, callback: HandlerCallback): HandlerCallback {
    // ...
  }

  /**
   * This registers the handler for auto-removal in the event
   * that the handler is re-registered since handlers can only be
   * registered once at a time.
   */
  private cleanupHandler(handlerName: string) {
    if (this.registeredHandlers.has(handlerName)) {
      this.unregisterHandler(handlerName);
    }

    this.registeredHandlers.add(handlerName);
  }

  private unregisterHandler(handlerName: string) {
    const canRemove = this.registeredHandlers.has(handlerName);

    if (canRemove) {
      this.registeredHandlers.delete(handlerName);
      // This calls ipcMain/Renderer.remove.<name>
      if (this.ipc.remove[handlerName]) {
        this.ipc.remove[handlerName]();
      }
    }
  }
}

An example impl:
export class AppIpcHandlers extends BaseIpcHandlers<typeof appIpcMain> {
  constructor() {
    super({
      logPrefix: 'App',
      ipc: appIpcMain,
    });

    // registers the handler when we start the app up
    this.bounceApp();
  }

  // this is if we need to register the handlers in an async fashion
  async initGlobalHandlers() {
    return;
  }

  bounceApp() {
    this.ipc.handle.bounceApp(async (_, { data: bounceType }) => {
      if (bounceType === 'critical' || bounceType === 'informational') {
        app.dock.bounce(bounceType);
      }
    });
  }
}

We create these classes and call await initGlobalHandlers() in our main/index.ts during app init


OSError: [Errno 121] Remote I/O error SH1106 display

Hello everybody,
I have a problem with the oled SH1106 module on raspberry pi 3B +.
When I try the examples, it tells me the following message:
Version: luma.oled 3.4.0 (luma.core 1.12.0)
Display: sh1106
Interface: i2c
Dimensions: 128 x 64
------------------------------------------------------------
Traceback (most recent call last):
  File "font_awesome.py", line 161, in <module>
    main()
  File "font_awesome.py", line 156, in main
    draw.text((left, top), text=code, font=font, fill="white")
  File "/usr/local/lib/python3.7/dist-packages/luma/core/render.py", line 43, in __exit__
    self.device.display(self.image)
  File "/usr/local/lib/python3.7/dist-packages/luma/oled/device/__init__.py", line 125, in display
    self.data(list(buf))
  File "/usr/local/lib/python3.7/dist-packages/luma/core/device.py", line 46, in data
    self._serial_interface.data(data)
  File "/usr/local/lib/python3.7/dist-packages/luma/core/interface/serial.py", line 119, in data
    write(list(data[i:i + block_size]))
  File "/usr/local/lib/python3.7/dist-packages/luma/core/interface/serial.py", line 128, in _write_large_block
    self._bus.i2c_rdwr(self._i2c_msg_write(self._addr, [self._data_mode] + data))
  File "/usr/local/lib/python3.7/dist-packages/smbus2/smbus2.py", line 637, in i2c_rdwr
    ioctl(self.fd, I2C_RDWR, ioctl_data)
OSError: [Errno 121] Remote I/O error

While with the SSD1306 screen, no display issues.
Has anyone ever had this problem?
Thanks by advance for your return,

Hi @Pada69
from the log output it seems that you may an issue with the i2c bus connecting the rpi with the oled screen, could you check with a linux i2c tool such as i2cdetect with such commands
i2cdetect -y 0
i2cdetect -y 1

and validate that the i2c address detected on the buses are the address of the sh1106 (it should be 0x3C from docs but maybe 0x3D). Can you print the result of the commands ?
Regards,
Xavier.

Hi @xcarcelle,
Thanks for your return. After more investigation, we use a shield on the raspberry pi who power supply display with 5V on VCC, SCL and SDA. When we have bought the diplay, it's was indicated 3.3 - 5 V  power range supply.
But the datasheet indicate Vdd1 1.65 -3.5V and Vdd2 3.0 - 4.2V.
When we plug the display with 5v and start an oled.examples, the beginning of the examples print on the screen but after 0.5 second, the error message appear. And when we add "device.persist = True" the screen keep the last image print on it.
We have solved the problem with power supply the screen with 3.3v on VCC, SCL and SDA.

We have solved the problem with power supply the screen with 3.3v on VCC, SCL and SDA.

Good to hear!


Wrong Return Type for BIO_ptr_ctrl

The return type of BIO_ptr_ctrl is according to every version of the documentation char*. However, in the header file (bio.h), the return type is void* (at least on master and in version 1.1.1).

I will take a look, it looks there is already a commit with a change in the documentation.

@Thomas1664 Did you want to submit your patch as a PR?

@Thomas1664 Did you want to submit your patch as a PR?

Yes but I didn't include that no cla is required in the commit message

You can fix that with git commit --amend and git push --force.

You can fix that with git commit --amend and git push --force.

@t8m Thanks. You will receive my PR in the next hours!


Reduced syntactic redundancy in Ravel.module by deriving module name from path

Fixes #29

Coverage remained the same at 100.0% when pulling 48c84597962d6e450c4fdb539c08c6e8cfdb0b9e on feature/29 into d5def2c7d957284622c981bc81f31891730935e4 on develop.

Coverage remained the same at 100.0% when pulling 1cf398fcda3f19cef48f34375631596f8f43d95c on feature/29 into d5def2c7d957284622c981bc81f31891730935e4 on develop.


Docs for find_peaks_cwt are incorrect

Docs for scipy.signal.find_peaks_cwt state that the default for gap_thresh is 2, but the code says (https://github.com/scipy/scipy/blob/master/scipy/signal/_peak_finding.py#L509)
if gap_thresh is None:
    gap_thresh = np.ceil(widths[0])

Thanks

@rgommers I wish to work on the issue. We have to update the docstring for gap_thresh, correct?

Since, this issue remains to be unsolved , I wish to claim this issue.

Commit d1ec9438284992fbac80299ced9c1e010bdad091 closes #6922

#6922 Commit  364aa77a52eac66a3c46485801334db9e9aef07a


new tutorial abap flp and update deployment tutorial

Do you need the new tutorial added to a group?

Do you need the new tutorial added to a group?

Hi Daniel, for now, it would be better not to add it to a group. When this tutorial is also possible in trial, we can add it.
Thank you.
Best regards,
Merve


chapter 8: “train_model.py” running failure issue

i am running the chapter 8 example. i use the argparse "python train_model.py --year 16 -r RUN"; The "train_model.py” is failure to run by the issue:
Traceback (most recent call last):
File "train_model.py", line 95, in
buffer.populate(1)
File "D:\Anaconda3\lib\site-packages\ptan\experience.py", line 365, in populate
entry = next(self.experience_source_iter)
File "D:\Anaconda3\lib\site-packages\ptan\experience.py", line 173, in iter
for exp in super(ExperienceSourceFirstLast, self).iter():
File "D:\Anaconda3\lib\site-packages\ptan\experience.py", line 53, in iter
obs = env.reset()
File "D:\Anaconda3\lib\site-packages\gym\wrappers\time_limit.py", line 25, in reset
return self.env.reset(**kwargs)
File "D:\pythoncode\Deep-Reinforcement-Learning-Hands-On\Deep-Reinforcement-Learning-Hands-On-master\Chapter08\lib\environ.py", line 302, in reset
self._instrument = self.np_random.choice(list(self._prices.keys()))
File "mtrand.pyx", line 1126, in mtrand.RandomState.choice
ValueError: a must be non-empty
please help me. thanks for your advices

Looks like you data is empty. Have you checked that --year arg has proper value?

yes, i find this empty issue. After read the book，i doesn't find the example argparse just like  "python train_model.py --year 16 -r RUN" . So i have the question what is the proper value of --year arg this code?

Looks like you data is empty. Have you checked that --year arg has proper value?

yes, i find this empty issue. After read the book，i doesn't find the example argparse just like "python train_model.py --year 16 -r RUN" . So i have the question what is the proper value of --year arg this code?


[CANNOLI-284] Fix release script, default is Spark 3 Scala 2.12 profile

Fixes #284

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/cannoli-prb/264/


Optimized the code

Simplified Heading Parsing: extract_headings checks for headings and depth levels in one pass, handling table of contents and code blocks directly.
Optimized format_heading: Reduced complexity by directly formatting links.
Efficient ToC Generation: generate_toc handles indentation dynamically, reducing nested conditions.
File Handling Streamlining: read_file and find_toc_bounds centralize file I/O operations for easier troubleshooting and consistency.
Concise Argument Parsing: Error checks are minimal and clear, ensuring only markdown files are accepted.
Overall Structure: Main function logic is linear, improving readability.

Hey, thanks for the PR! Please format this to match the PR template and show how you have tested to make sure everything still works as expected 😄


Redirect has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header errors

See comment here: https://github.com/gijzelaerr/js9notebook/issues/2#issuecomment-402205926
This only happens when trying to load up js9 via Jupyter's web server (via /files/xxx).
Workaround is to start jupyter notebook with the --NotebookApp.allow_origin='*'. But I'm probably committing some sort of horrible security sin here (@gijzelaerr?) I really don't understand enough about it to judge...

Caused by #7.


fix(helm): fix helm history unit tests

fixes #3652

Hi @adamrees
I am curious to know if the history changes broke test or its more restructuring that you are doing
Your feedback will help me for future contributions

@rajatchopra It was a rebase/merge issue. This is to fix master which is currently broken https://circleci.com/gh/kubernetes/helm/5405


OOM on the mbuf freelist (100+GB)

the nutcracker process consumed 100+GB phisycal memory on my production box after a data migration from another redis to this one (nutcracker).
and the gdb console showed below:
(gdb) p nfree_mbufq
$1 = 6365614
(gdb) p mbuf_chunk_size
$1 = 16384

the memory consumption was nfree_mbufq * mbuf_chunk_size = 101GB approx.
I have read some code fixes (pr)s about the similar phenomenon, like:
https://github.com/twitter/twemproxy/pull/461
https://github.com/twitter/twemproxy/issues/203
but such fixes didn't set the limitation of the mbuf chunks, so the OOM was still here,
I coded a fix, and the nutcracker can pass a command param ('-n ',  in my fix) to set the max number of mbuf chunks, once exceeded the limitation it can free the mbuf after processing one req immediately.

Hi, can you share your fix. I am facing the same issue.

I mergerd the code fix onto my repo:
https://github.com/inter169/twemproxy/commit/2b2a0d0cfdbb18d77b510a86283943eafa798d94

@inter169 Memory can not be freed after even free all mbufs but free msgs together in my test. I think the reason is syscall brk in malloc can not free memory fragment to os.

https://github.com/twitter/twemproxy/pull/486 was merged in 2016 so 461 isn't relevant.
I'd agree that some sort of memory limit would make sense, and that mbufs aren't freed
During pathological networking situations (e.g. limited bandwidth, extremely high timeouts and high traffic) high memory usage can be a problem


RAT-384 naming changes (RAT-384 Part 1)

Adjusts the Maven names of Options and provides standard output for the naming across supported UIs.
This is preparatory to being able to generate the proper Maven help for RAT-384.

Changes look good to me.  Though my plan is to do a massive clean up of checkstyle issues after Harmonization (or at the end of harmonization)

@Claudenw if you are okay with my changes feel free to merge this PR. Thanks again for the changes!


Supporting Minio (or other non-AWS S3 solution)

I'd like to use OC with minio. Since it's S3-compliant, it could easily be possible by allowing configuration of the S3 API endpoint (as opposed to the AWS endpoints). Minio also has a requirement of using v4 of the S3 signature algorithm.
var s3  = new AWS.S3({
          accessKeyId: 'YOUR-ACCESSKEYID' ,
          secretAccessKey: 'YOUR-SECRETACCESSKEY' ,
          endpoint: 'http://<IP_ADDRESS>:9000' ,
          s3ForcePathStyle: 'true', // needed with minio?
          signatureVersion: 'v4'
                                });

http://docs.minio.io/docs/how-to-use-aws-sdk-for-javascript-with-minio-server

Looks like a conditional update to these lines would do the trick!

Hi, I made actually a bit of research. OC uses file-based ACLs to secure some files, and unfortunately minio doesn't seem to support that (yet): https://github.com/minio/minio/blob/77a192a7b560f4f7c933280536231996cfb39414/docs/minio-limitations.md#list-of-amazon-s3-object-apis-not-supported-on-minio
I think minio looks like a great project and perhaps it would be possible to ask for that feature. In any case, I would be happy to accept a PR for OC to make additional S3 parameters configurable, such as endpoint, signatureVersion, etc. Would you be able to try?
Only thing, I removed the minio suggestion from the wiki until we ensure it allows full integration - we can add it later.

Sure thing! Once I dive in a bit deeper, I'll see if I can get a PR made.

I am closing this at the moment. We can reopen in case we still are interested.

I was just researching this aspect of S3 API support.  We have some fine-grained access control requirements and are looking for open source solutions that might form a back-end for our tools.  We have a draft requirements document here

I have created a PR #662 against this issue.  I created minio in my kubernetes cluster (on GCP) with no external access, so this should minimize security concerns.
The plan would be that it supports the following s3 config.  I could create a minio flag and set the s3ForcePathStyle & s3ForcePathStyle in the code.  But since these are valid s3 options, figured I would leave them configurable.  Thoughts?
s3: {
key: 'YOUR_KEY',
secret: 'YOUR_SECRET_KEY,
endpoint: 'http://minio.default.svc.cluster.local:9000',
path: 'http://minio.default.svc.cluster.local:9000/YOUR_BUCKET/',  //Now supporting protocol
s3ForcePathStyle: 'true',
signatureVersion: 'v4',
componentsDir: 'components',
bucket: 'YOUR_BUCKET',
},

I have closed my pr due to that fact that I now understand that oc requires most files to be public.


Don't generate interface properties for customized interface properties

Hi!
So, related to https://github.com/limbo-works/Limbo.Umbraco.ModelsBuilder/issues/3... Sometimes I customize the property for an interface. For instance, Content Pickers or MNTP by default will be returning "IPublishedContent", but I might know that it will always be of a specific ContentType . In order to be able to call the property and get the strongly-typed model in a View, I customize the composition model and interface. Example:
Customized 'CompRelatedBlogPosts.cs'
partial interface ICompRelatedBlogPosts
    {
        IEnumerable<BlogPost> SelectedPosts { get; }
    }

    partial class CompRelatedBlogPosts
    {
        [ImplementPropertyType("SelectedPosts")]
        public IEnumerable<BlogPost> SelectedPosts => GetSelectedPosts(this);

        public static IEnumerable<BlogPost> GetSelectedPosts(ICompRelatedBlogPosts that)
        {
            return that.Value<IEnumerable<IPublishedContent>>("SelectedPosts").ToContentModels<BlogPost>(true);
        }
    }
}

Then I just add an override to set the correct type in the Models which use the Composition. Example:
partial class TextPage
    {
        #region Implementation of ICompRelatedBlogPosts
        IEnumerable<BlogPost> ICompRelatedBlogPosts.SelectedPosts => CompRelatedBlogPosts.GetSelectedPosts(this);
        #endregion
    }

This works fine. However, when I re-generate the Models, the generated composition file gets the "IPublishedContent" property again:
'CompRelatedBlogPosts.generated.cs'
 public partial interface ICompRelatedBlogPosts : IPublishedContent {

        System.Collections.Generic.IEnumerable<global::Umbraco.Cms.Core.Models.PublishedContent.IPublishedContent> SelectedPosts { get; }

    }

I can solve the issue by explicitly ignoring the property:
'GetModelsNotificationHandler.cs'
    public class GetModelsNotificationHandler : INotificationHandler<GetModelsNotification>
    {
        public void Handle(GetModelsNotification notification)
        {
            foreach (TypeModel model in notification.Models)
            {
            ...
				//Content-Type specific processing

				if (model.Alias == CompRelatedBlogPosts.ModelTypeAlias)
				{
					foreach (PropertyModel property in model.Properties)
					{
						if (property.Alias == "SelectedPosts")
						{
							property.IsIgnored = true;
						}

					}
				}
               ...
            }
        }
    }

But I wonder if the generating code can check for the customized property defined in a custom interface implementation and skip it, like it does for the regular properties?

If the picker is a MNTP, it might also be worth looking at our custom MNTP package.
It let's you select your custom "item converter" in the data type, which could then return the selected blog posts as BlogPost instead of IPublishedContent. Then the property will return the correct type regardless of whether you're using ModelsBuilder or calling the .Value() method directly.
If I implement #5, it would also mean that property values are better cached compared to when handling the conversion directly in the model.

I still like your proposal, so the above is just an example on another approach with the same end result 😉

@abjerner Cool package! Right now I'm working with a migrated site with a lot of standard Content Pickers in use, so it wouldn't really help all my use-cases.
Also a question - it seems your package was specifically designed such that you would manually design the data models and set up the Converter. Does it handle a basic usage of just using the standard Model for a specific existing DocType?
Regarding OMB's handling... What WAS handled by OMB was that if I "redefined" a property in an interface, it would skip re-generating it in the Composition model. What it never id d support was automatically adding the derived-models customization
In my example, this bit I always had to add myself to all the affected Models:
partial class TextPage
    {
        #region Implementation of ICompRelatedBlogPosts
        IEnumerable<BlogPost> ICompRelatedBlogPosts.SelectedPosts => CompRelatedBlogPosts.GetSelectedPosts(this);
        #endregion
    }

Sometimes that could get annoying - if I had a Comp which I used on a lot of Doctypes, I would have to create "Custom" partials for all those Doctypes - even if this was the only customized bit. But, it's a pretty simple copy/paste for the code snippet, so not a huge deal.

Hey @hfloyd
Just to let you know that I haven't forgotten you, but I haven't had the time yet to look through your issues. I hope I will have that soon 😉

No problem, @abjerner I know you have other things to work on also. I like to add issues when I come across them, but certainly have no expectations for instant attention. 😊


Issue in pulling Azure platform audit logs in Splunk

Hello everyone,
Vontier InfoSec team needs your help in fixing an issue we’re currently facing when trying to pull Azure audit logs in Splunk.
The integration to pull this specific data is configured on Splunk side through the Splunk Add-on for Microsoft Azure (Version 4.0.3, Build<PHONE_NUMBER>) and looking at the integration logs on Splunk side, I have found a specific error (see attachments).
This integration is not new. It did work well for months. Then suddenly it broke.

VNT InfoSec and VNT Cloud teams did dig deeper and we think we have found the root cause of this issue.
Looking at the logs, the Graph API call we saw is the following.
https://graph.microsoft.com/v1.0/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+2022-06-07T06:44:54.23518Z+and+activityDateTime+le+2023-05-26T13:19:07.832439Z
Now in the input config dialog box if no value is given in the “Start Date” field the input will start getting data 7 days in the past.
But looking again at the start date in the query above you can see that the start date is in the 2022!

Then we tried to query the Azure platform using the MS Graph explorer tool and we used the same query but with a proper start date. See below the query used in Graph Explorer.
https://graph.microsoft.com/v1.0/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+gt+2023-04-25T06:44:54.23518Z+and+activityDateTime+le+2023-05-26T13:19:07.832439Z

Using this tweaked query, we obtain audit data from Azure (see attached picture).

So the root cause is the Start Date value that is wrong compared to the settings in the input config field (see attached picture).

Furthermore, we tried to enter the good Start Date we used in Graph Explorer, but the dialog box didn’t accept the setting despite is correct (see attached picture).
Maybe another bug?
All the findings have been shared with Splunk support. They suggested to submit an issue to the developer.
We need your help to review/fix this issue to restart pulling this data into Splunk.
Please let me know if you have any comments, questions, or concerns.
Thanks.
Regards.
Massimo Morsiani - VNT Information Security

Sorry I forgot to attach the error in the dialog box I mentioned.

Hello everyone,
any feedback about this issue?
Thanks.
Regards.
Massimo Morsiani - VNT Information Security

This looks like an old checkpoint value was picked up.  The checkpoint holds the last date/time read as the starting point for the next query.  Checkpoints key off of the input name.  So, if a new input was created that reused an old input name, an old start date could be picked up.
Checkpoint values can be observed by navigating to Troubleshooting > View Checkpoints within the add-on.
Setting a start date in the input will not override the checkpoint start date.  Here is a link to the specific logic => https://github.com/splunk/splunk-add-on-microsoft-azure/blob/8330f66d155af431fb4e704239d8f12517b68785/package/bin/MS_AAD_audit.py#L40
Creating a new input with a new name should correct the issue.


support pytest-xdist

@prashanth-sams
if you can summarise the xdist run it would be good.. it comes as individual build..
for 53 cases run parallely by pytest -n=2 it triggers indviudal build..
for the 53 cases run , out of which 40 passed and 13 failed.. it created many builds
i cannot see the status of 53 cases at one place.. hope you are aware about this

Hi @prashanth-sams, is there a workaround for this or this will be still in progress?
Thank you


New version: Kopia.KopiaUI version 0.11.1

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.1 schema?

Microsoft Reviewers: Open in CodeFlow

@wingetbot run


Running out of ram

I couldnt get HTK to work properly, possibly due to bad installation. But seamed to work fine with librosa.
However when it comes to '===> Reading audio files... it seams like the for loop going over the audio paths just fills up my  8gb of ram and swap. And this is only on the 28539 files from train-clean-100. And it doesn't produce any files at this stage.
Is there a trick I am missing to get the preprocessor going without reading everything into ram all at once?
Eta was over 1 hour and it broke down after 54% of the train-clean-100 dataset.

Hi, @jesuistay
Only 1 file will be loaded In each loop for the memory efficiency, so I don't know why.
Which loop do you mean?
There are 3 loops in librispeech/inputs/input_data.py.

First one: for i, audio_path in enumerate(tqdm(audio_paths)):
To me it looks like it traverses the entire dataset and creating the dict, in order to calculate the mean std I assume.
For now I've just skipped the all the normalization and just writing the npy after I get the input_data_utt from librosa.

I managed to get HTK working, but the ram problem still confuses me. I had to increase my swap partition to 16 gb (+ 8gb of ram) just to manage and preprocess the clean-100


[crash] Something went wrong

Describe the bug
OS: ubuntu 20.04
Steps to reproduce
Stacktrace
Cannot set properties of undefined (setting 'cluster')
The error may be correlated with this previous error:
TypeError: Cannot set properties of undefined (setting 'cluster')
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:334304:21
   at Array.forEach (<anonymous>)
   at Function.unflatten (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:334273:23)
   at mergeConfigs (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:61392:237)
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:61033:3744
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60956:25
   at memoized (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60864:20)
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60970:40
   at Object.memoized [as current] (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60864:20)
   at Object.checkForUpdates [as onStateChange] (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:635653:48)

TypeError: Cannot set properties of undefined (setting 'cluster')
The error may be correlated with this previous error:
TypeError: Cannot set properties of undefined (setting 'cluster')
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:334304:21
   at Array.forEach (<anonymous>)
   at Function.unflatten (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:334273:23)
   at mergeConfigs (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:61392:237)
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:61033:3744
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60956:25
   at memoized (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60864:20)
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60970:40
   at Object.memoized [as current] (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60864:20)
   at Object.checkForUpdates [as onStateChange] (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:635653:48)

   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:334304:21
   at Array.forEach (<anonymous>)
   at Function.unflatten (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:334273:23)
   at mergeConfigs (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:61392:237)
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:61033:3744
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60956:25
   at memoized (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60864:20)
   at file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60970:40
   at memoized (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:60864:20)
   at useSelectorWithStoreAndSubscription (file:///tmp/.mount_MonoklW7iGa6/resources/app.asar/build/bundle.main.js:635620:30)

Hi @nguyenhoangnam3111996 , thank you for this crash report!
Could you tell us when has this happened or if you have a rough idea on how we can reproduce it?

it happened when i tried to open folder with many subfolders in it

Hi @nguyenhoangnam3111996, we released the 1.7.5 version of Monokle which includes a fix for this error.
You can download it here: https://github.com/kubeshop/monokle/releases/tag/v1.7.5
I'm closing this issue but please let us know if the problem persists after 1.7.5.
Thank you!


built-in support for offline-first projects: record/table/global stamps, merging, syncing

Is your feature request related to a problem? Please describe.
nope
Describe the solution you'd like

every mobile app we have written assumes that the network connection will be intermittent. that means that each client has its own sqlite instance, followed by syncing with the server.
obviously, some operations must be done, live, and rejected, if not
to facilitate this, we have implemented record stamps, table stamps, and global stamps on our databases. when the mobe has comms with the server, syncing and merging occurs.
serverpod supporting + automating this process would be a great

Describe alternatives you've considered
implementing these features, ourselves.
Additional context
Add any other context or screenshots about the feature request here.

Hey @macMikey!
This is a feature we are really enthusiastic about and have on our long-term roadmap. It is quite a big feature to build that will take some careful consideration to integrate in an intuitive way.
But if everything goes as planned, this should be part of the Serverpod framework in the future.
We already have a couple of other issues that track somewhat related features: https://github.com/serverpod/serverpod/issues/2258, https://github.com/serverpod/serverpod/issues/1719
Also worth mentioning this discussion: https://github.com/serverpod/serverpod/discussions/2084
Do you think this feature is represented in the other issues or should we track this separately here?

oh, look, crdt.
never mind, closing this issue.


Implement face alignment

https://github.com/akshitagupta15june/Face-X/tree/master/Awesome-face-operations

Hey @akshitagupta15june, I would like to work on this issue. Can you please tell me ways to get started?

Hey, I would like to work on this issue. Can I do it?

Yes you can


refactor(SetTheory/Ordinal/Basic): type_lt → type_toType

The new name is much more discoverable and makes clear that this isn't just any < relation.

maintainer merge

bors merge


Add colcon autocomplete to Dockerfile

For #49, Still not sure if this is the right solution

Closed since https://github.com/athackst/dockerfiles/pull/266 has been added to the base image


BRAW shots suddenly just colored solids

Hi there,
Love the plugin so far, but having another issue. I'm hoping there's an easy fix.
Quite suddenly, all of the braw files no longer display video. Instead I get a solid color which varies in brightness as I scrub through the footage (see screenshot). No updates or changes to the system were made -- though I've since updated and reinstalled everything. No change. I also tried transcoding to mov. Mov files import fine, but the "Convert BRAW clips to MOV's within an event" tool processes my footage, but won't add it back to my library. I don't get an error, it just does nothing as I would expect if I were dragging to an event.
If I create a brand new library and import a braw clip using braw toolbox, it comes in as colored solids.
Ventura 13.6
FCPX 10.6.10
BrawToolBox 1.3.2 (57)
Please help! Project is stalled and missing deadlines.

Just to confirm, if you try and import a BRAW clip in a completely new Library via the Workflow Extension, it just comes across as a solid colour? What happens when you step inside the Multicam clip, then step inside the Synchronised clip and modify the BRAW Toolbox effects attributes like Exposure?

1.3.2 (57) - re-ran the import test, same result. Adjustments to effects attributes work as you would expect. I.E. if you increase exposure, the solid color gets brighter/moves toward white.
Upgrading to 1.3.3 (59) we're now getting the "this plugin is not responding" error. If we can get past that, I'll let you know.
Unfortunately, we don't have a backup of 1.3.0. Is that something we can download from github somehow?

If it doesn't, I've been trying to help another user that's having some weird problems on an older Intel Mac, so if you send me your email address, I can try ad you to that TestFlight build.

Happy enough to send along my email, but would prefer not to post it to github. What's the best method of getting it to you?

When you open the main BRAW Toolbox application - are all the latest Motion Templates installed?
See: https://brawtoolbox.io/installation/

1.3.2 (57) - re-ran the import test, same result. Adjustments to effects attributes work as you would expect. I.E. if you increase exposure, the solid color gets brighter/moves toward white.

That's super, super weird. I've never seen anything like that before and not even sure how it's even possible.

Upgrading to 1.3.3 (59) we're now getting the "this plugin is not responding" error. If we can get past that, I'll let you know.

It would be great if you could send me any crash logs, along with new copies of the log files you've already posted above.
See: https://brawtoolbox.io/troubleshooting/#ive-run-into-a-bug-where-can-i-find-the-log-files

Happy enough to send along my email, but would prefer not to post it to github. What's the best method of getting it to you?

You can contact me via: https://brawtoolbox.io/support/

That's super, super weird. I've never seen anything like that before and not even sure how it's even possible.

Not sure if it helps, but the color appears to come from the video. Almost like it's just grabbing one pixel and making it full screen.

Super strange! So it was working fine, then one day it just started doing this?
Have you tried trashing FCPX's Preferences?

Have you tried trashing FCPX's Preferences?

Oh yes. Repeatedly. :( I'm out of the office today, but will try to get you additional logs and such first thing tomorrow.

Super strange! So it was working fine, then one day it just started doing this?

Yeah. I was tweaking the edit on a scene and decided to change out one take for another. All of them were in the bin already and had been for weeks at this point. I selected my in and out, drug it to the timeline and it was just green. No video. I started troubleshooting and finally, deleted all the existing render files. At that point EVERY clip was just a solid color. So, I don't know when it broke. Apparently, I'd been floating along on the render cache for some period of time.

Actually, I bet that's the answer. I bet it was working on 1.3.0 and we didn't spot that it had broken with the update, because the render cache kept showing us working footage.

Given this issue is now a month old, I'm going to close this for now, but feel free to re-open if you're still having issues or if you have any questions.
If you're still having issues with BRAW Toolbox v1.3.4 (62) please let me know.

Thanks, Chris. I saw the new beta was out and haven’t had a chance to try it yet. But based on release noted it does sound like it will get me sorted. Thanks so much for all your help!Sent from my iPhoneOn 16 Nov 2023, at 22:55, Chris Hocking @.***> wrote:
Given this issue is now a month old, I'm going to close this for now, but feel free to re-open if you're still having issues or if you have any questions.
If you're still having issues with BRAW Toolbox v1.3.4 (62) please let me know.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>


Needs implementation for --ext-code and --tla-code

These were on my backburner, because I thought I didn't need them, but I ran into a situation today that required it. This issue is mostly for my benefit (I'll try to whip one out this weekend)

I too noticed that these are missing (along with their "file" variants). @anguslees, is their omission deliberate, or do you think it would be useful to have these available in kubecfg?


Solved 4 ISSUES: I have modified footer section, added css to the social media icons, formatted the nav-links in the header section, added apt css for welcome page , unnecessary message box below the form (above the footer) was removed.

I have added proper css for footer, social media icons, enabled the light-dark mode button and formatted the buttons for all navigation links present near the header.Also I removed unnecessary message box at the bottom, which was repeated twice , once in the form and the other below it. So, I removed the message box present below the form to avoid confusion. A total of 5 issues solved.
Title of the Pull Request

Related Issue
Issue number: 524
Issue number: 545
Issue number: 513
Issue number: 555
Description
For issue number 524: Updated the CSS of the footer section.
For issue number 545: Alignment of buttons in the home section have been made.
For issue number 513: Duplication of message box has been removed
For issue 555, I added a decent CSS to the header section to look elegant.
How Has This Been Tested? ⚙

Screenshots 📷
A) For ISSUE 524:

B) For ISSUE 545:

C)For ISSUE 513:

D)For ISSUE 555:

Type of change
What sort of changes have you made:
I have modified the footer section, added CSS to the social media icons, formatted the nav-links in the header section, added apt CSS for the welcome page, and unnecessary message box below the form (above the footer) was removed.
Checklist

[x] My code follows the code style of this project.

[x] My code follows the code style of this project.

[x] I have followed the contribution guidelines

[x] I have performed a self-review of my own code.

[x] I have ensured my changes don't generate any new warnings or errors.

[x] I have updated the documentation (if necessary).

[x] I have resolved all merge conflicts.

Please resolve merge conflicts

Please resolve merge conflicts

Okay after resolving merge issues its asking me to commit merge with 2 options 1)1) Commit updates to the main branch.2)Create a new branch and commit updates. Your pull request will be updated automatically. Which option shall I choose ?

@vishanurag Completed resolving conflicts, kindly do check and please assign me as level-3 because I solved 4 issues so far

I just tested your changes in codespaces and here is the result.
https://github.com/user-attachments/assets/dffd5c83-ac7d-4af8-8511-c9b8dada0bac
I'm rejecting your PR this time because it will make the entire project non-responsive also a lot of styling will be gone.

But when I executed it these were not the results. They were the ones that I attached in the screenshots previously. I don't understand why it happened. A lot of efforts have gone unrecognized :-(
Let me try to resend my pr once again, if the same happens you can reject it. Please try to understand sir. Kindly give me an opportunity.

Ok

https://github.com/user-attachments/assets/009a1cfe-7235-4d52-81f7-af14cf4a03a8
@vishanurag I have executed just now and here's the result. No styling has gone out of order. So, I will create a new pr and resend it to you.

@vishanurag just check out pr no #1027. I have created a new pr and resent it
Thanks and regards.
Hopefully, it works this time as expected


Update deprecated parameters use

This change was made from hugo's version 0.112.0 "The use of site.Language.Params is deprecated. Use site.Params directly"
More info here: https://gohugo.io/content-management/multilingual/#changes-in-hugo-01120

Sorry it took me such a long time to get to your PR. For some weird reason I wasn't watching my own repo 😅 Thank you very much for the contribution!

No worries! Your project is awesome! Happy to help!


support bad pack ids whitelist for identity name validation

instead of https://github.com/coda/packs/pull/2681, let's just whitelist all identity names from the SFDC pack to avoid breaking existing enterprise users.

@chris-codaio hmm, I maybe to do that but will need help on a few things:

need SRE help to figure out all the affected docs.
need advice to how change the sync table key in a doc. @jonathan-codaio how do we do this? I suppose this is different from recreating a table but needs to manipulate the table key live right?


Could not get access token: The user may have revoked the authorization response from LinkedIn.com was empty.

Q
A

Bug?
no

New Feature?
no

Are you using composer?
yes

Version
Specific version or SHA of a commit

Actual Behavior
What is the actual behavior?
Im trying to login using the example code given and everything is ok but after accept app to use my linkedin information this error appears, i read a lot but i couldn´t find a solution
Expected Behavior
What is the behavior you expect?
Obtain access token and successful login
Steps to Reproduce
im using laravel 5.3 and the following code in one of my routes
$linkedIn=new Happyr\LinkedIn\LinkedIn('CLIENT ID', 'SECRET ID');

if ($linkedIn->isAuthenticated()) {
    //we know that the user is authenticated now. Start query the API
    $user=$linkedIn->get('v1/people/~:(firstName,lastName)');
    echo "Welcome ".$user['firstName'];

    exit();
} elseif ($linkedIn->hasError()) {
    echo "User canceled the login.";
    exit();
}

//if not authenticated
$url = $linkedIn->getLoginUrl();
echo "Login with LinkedIn";

thanks !!

How did you solve your issue?

The problem is solved, it was related with this error "cURL error 60: SSL certificate problem: unable to get local issuer certificate (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)"
So i add a cacert.pem downloaded from https://curl.haxx.se/ca/cacert.pem into php.ini
; A default value for the CURLOPT_CAINFO option. This is required to be an
; absolute path.
curl.cainfo = "absolute route in server"

and problem solved. thanks

Awesome. Thank you for reporting this!

Hello Nyholm
I also got following error -
"Could not get access token: The user may have revoked the authorization response from LinkedIn.com was empty. "
I read a lot. But not find answer.  Could you give me sample code, how to get access token from 'code' (after user allow access) and so I can store in db..
For reference I have attached my code snippet here.
Could you please help me in this
My symfony2 application code is below--
`/**
* @Route("/linkedin/callback", name="social_linkedin_callback", options={"expose"=true})
* @Method("GET")
* @param Request $request
* @return array
*/
public function linkedinCallbackAction(Request $request)
{
$session = new Session();
$params = $request->query->getIterator()->getArrayCopy();
$responseData = [];
    //get session state
    $oauthState = $session->get('oauth_state');

    //check does 'state' param is same which we set in session
    if (isset($params['state']) && $oauthState == $params['state']) {

        $linkedIn = new LinkedIn($this->container->getParameter('linkedin_credential')['consumer_key'],
         $this->container->getParameter('linkedin_credential')['consumer_secret']);

        //here error comes when calling getAccessToken method
        $accessToken = $linkedIn->getAccessToken();

        //access token is not get from this method
        //Todo: Save access token in db

    } else {
        $responseData['success'] = FALSE;
        $responseData['message'] = 'Invalid request, State is not matched. ';
        $session->set('socialResponse', $responseData);
    }

    return $this->redirect($this->generateUrl('social_setting', array(), true), 301);
}`


Multi class and Multi label segmentation

What does this PR do ?
This PR makes it possible to use Multi-Class Segmentation and Multi-label Segmentation. Furthermore, several losses have been added for unbalanced segmentation tasks.
Collections:
MTL
Changelog

There are too many changes to add specific lines

Usage

See train/mtrls.yaml for usage within the model
see tests/segmentation/losses for using the separate losses

Before your PR is "Ready for review"
Pre checks:

[x] Make sure you read and followed Contributor guidelines

[x] Did you write any new necessary tests?

[] Did you add or update any necessary documentation?

[] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)

[] Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

[x] New Feature
[ ] Bugfix
[ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information

Related to # (issue)

@TimPaquaij please also check the CodeCov/Tox errors. It seems like there is an issue with importing the new losses.

I think it is not valid to make a new collection, since that would suggest that this is a different task. Whereas I think it is a different approach to achieving the same task. For some datasets, the multi-label approach would be more beneficial, think of detecting strokes within the brain. In this case, you have 2 labels which can be both valid. One pixel can be brain and there could be a lesion. However, even for this dataset, you can also determine the areas of the lesions with a multi-class approach since eventually, we are interested in the locations and the size of the specific lesions.

So to be clear, the original segmentation method you implemented was based on a multi-label segmentation (MLS) approach. Where, each class is handled separately and is not influenced by another class (see thresholding in inference_step). For the SKMTEA dataset and for the upcoming spatially adaptive semantic guidance module (SASG), the segmentation needs to be in a multi-class segmentation form where the classes are 'related' (and can be transformed into probabilities). So this is why I wanted to have a variable which can be set to a specific segmentation approach. Since then I can use this variable later on to create warnings when different techniques are combined which can not be combined.

So to be clear, the original segmentation method you implemented was based on a multi-label segmentation (MLS) approach. Where, each class is handled separately and is not influenced by another class (see thresholding in inference_step). For the SKMTEA dataset and for the upcoming spatially adaptive semantic guidance module (SASG), the segmentation needs to be in a multi-class segmentation form where the classes are 'related' (and can be transformed into probabilities). So this is why I wanted to have a variable which can be set to a specific segmentation approach. Since then I can use this variable later on to create warnings when different techniques are combined which can not be combined.

This is also different from combining the classes like in the process_segmentation_labels, right?

So multi-label and multi-class is a type of approach. So it includes the way you should pre-process your data, determine the loss and asses the final prediction. In the case of multi-class segmentation it is not possible that an pixel is defined as no class. So we need to make sure that the background/not a class is also defined as an seperate class. Whereas the multi-label does not need this and can have the same number of classes

So multi-label and multi-class is a type of approach. So it includes the way you should pre-process your data, determine the loss and asses the final prediction. In the case of multi-class segmentation it is not possible that an pixel is defined as no class. So we need to make sure that the background/not a class is also defined as an seperate class. Whereas the multi-label does not need this and can have the same number of classes

FYI when I start training for MLS/MCS I get the warning about the weights. Maybe it has to do with the yaml file you shared with me but please confirm this is intended and doesn't break anything for MLS/MCS.

[atommic W atommic_logging:336] atommic/atommic/collections/multitask/rs/nn/base.py:184: UserWarning: Sum of segmentation losses weights is not 1.0. Adjusting weights to sum up to 1.0.
warnings.warn("Sum of segmentation losses weights is not 1.0. Adjusting weights to sum up to 1.0.")

@TimPaquaij I tried to implement MLS/MCS for single-task segmentation models and I found when computing the loss in process_segmentation_loss it will return a tensor of shape == len(classes), meaning the loss of each class. So then it will fail to get the loss.item() because it doesn't contain only one value.
Then I checked back in the MTLRS collection the same function and I saw that it will first return the loss tensor of shape == len(classes) and then it will select only the first class loss due to the hardcoded fix loss = loss[1][0] for the base dice loss we had before. Where loss[1] is the dice loss and not the dice metric (which is the loss[0]) and loss[1][0] is just the loss of the first class.
Could it be that now this is not implemented in the right way?

I think this is the result of me changing the batch size argument within the dice. In de docstring it says use this argument to get batch_size, whereas in de docstring of the dice loss on monai is was inverted. So what I did is inverting the logic by adding 'not' infront of each statement to get a correct logic with the docstring. When training place make sure to double check the parameters of the dice loss. Before you had batch_size on true and reduction on mean_batch, which resulted in a singels loss value. Since I changed the batch argument this now results in a loss with len(classes). So make sure to change the reduction to mean (reduction of both batch and channel). Or change the batch_size to false and the reduction to mean_channel. Please view test_dice to see the possibilities.
Why I changed this into loss[1][0] is because after a valid reduction the single loss of the dice is still in a list, which is fine when using only one loss but for combining it with other segmentation losses you will get an error

This resulted from changing the batch size argument within the dice. The docstring says to use this argument to get batch_size, whereas, in the docstring of the dice loss on monai, it was inverted. So, I did invert the logic by adding 'not' in front of each if statement to get the correct logic corresponding to the docstring. When training, please make sure to double-check the parameters of the dice loss before you had batch_size on true (resulted in the reduction of the batch_size) and reduction on mean_batch (which resulted in the reduction of the channel size since this was the first dimension after the batch_size reduction), which resulted in a single loss value. Since I changed the batch argument, this has now resulted in a loss with len(classes). So make sure to change the reduction to mean (reduction of both batch and channel). Or change the batch_size to false and the reduction to mean_channel. Please take a look at test_dice to see the possibilities.
Why I changed loss[1] into loss[1][0] because, after a valid reduction, the single loss of the dice is still in a list, which is fine when using only one loss, but when combining it with other segmentation losses, you will get an error

So dice_loss_reduction and dice_metric_reduction should be always set to mean instead of mean_batch from now on? Also are we sure the loss[1][0] doesn't break previous and single-task segmentation?

Yes indeed, all the reductions need to be set to 'mean' instead of 'mean_batch' when batch_size is set to 'true'!

Yes indeed, all the reductions need to be set to 'mean' instead of 'mean_batch' when batch_size is set to 'true'!

You mean dice_loss_batch and dice_metric_batch right? We really need to right down all those things upon merging the PR.

Okay I changed loss[1][0] into loss[1].squeeze() This makes sure if there is a batch or a channel dimension it does not interfere.This only makes sure when the loss is equal to a 0 dimension loss the loss --> torch.tensor([0.123]) will be transformed into torch.tensor(0.123) needed to combine with other loss. I think this is a better implementation


Configurable behavior when Frigate itself fails requests

Apologise if i missed anything. Normally, when frigate card loses connection, it retries to connect. Although i also have quite some times where i see some code of frigate being unavailable, and then it does not retry. even though when i restart the home assistant app, it immediately works again.
Is there perhaps something to be done about this?
Thanks!

here is one of the errors. again this goes away if i restart the home assistant companion app:

is it possible to send a personal message somehow @dermotduffy ? i captured a video, also about the card causing a crash of home assistant pretty reliably. but because it contains some data and imaging about our home i prefer to keep it private.

@borgqueenx I am dermotduffy on discord, feel free to message me there.
The photo shows a failed request to Frigate, but not a failed connection to Home Assistant. Failures of the latter kind (to Home Assistant) should always retry. Failures from Frigate currently do not retry.
If you are having regular Frigate failures, it might be worth understanding why that is happening (e.g. either Frigate logs, or Home Assistant logs may show something). If the card is sending too much traffic to the backend, this can also be throttled (see https://card.camera/#/configuration/performance).
If may be possible to add retry support or perhaps "ignore" support for some or all of these failures, but the consequences are probably going to be that the card appears either "hung" (for retries) or "empty" (for ignore, e.g. no snapshots shown at all when you visit the snapshots view).

here is another error that was given today:


fix appveyor to actually run dub

Apparently the variables APPVEYOR_JOB_ARCH wasn't defined and thus the dub test wasn't running.

Now it correctly checks and yields #109.

I like this one https://drive.google.com/folderview?id=0BwdiZp7qSaBhdFlCaTdhZlotYXM&usp=drive_web
but with completely different background and colors

@9il - this PR fixes our appveyor windows setup ;-)
2. link doesn't work :/

@9il - this PR fixes our appveyor windows setup ;-)

if "%PLATFORM%"=="x64" ( dub test --arch=x86_64 )
Generating test runner configuration '__test__d_source_library__' for 'd_source_library' (sourceLibrary).
Performing "unittest" build using dmd for x86_64.
mir 0.13.0+commit.15.g107f366: building configuration "__test__d_source_library__"...
Linking...
Can't run '\bin\link.exe', check PATH

Can't run '\bin\link.exe', check PATH

Do you have a 64bit Windows somewhere around to test?
My VM is only 32bit.

no

I hope that is all that is needed - will check

@9il

I like this one https://drive.google.com/folderview?id=0BwdiZp7qSaBhdFlCaTdhZlotYXM&usp=drive_web
but with completely different background and colors

To which one were you referring?

dscience_delta_upper_creative.svg.png

dscience_delta_upper_creative.svg.png

that's my favorite too ;-)

but with completely different background and colors

the default colors or something completely different?

the default colors or something completely different?

completely different, with different form


n_entities NOT found

Hi,
I found that the variable n_entities is not be initially assigned. There will be a error when the class QADataset is constructed.


chore:enable asynchronous execution

Enables asynchronous blocks execution when syncing. The functionality was already implemented but it was not used when syncing.

prnet cleaned-up successfully


[Host.Outbox] Fails to retrieve bus when no child buses are in use

When SlimMessageBus is configured to use a primary (Main) bus only, the OutboxSendingTask fails to locate the bus.
I have narrowed the issue down to the OutboxSendingTask.GetBus method but unfortunately don't have the time to dig/resolve the issue for the time being.
    private static IMasterMessageBus GetBus(ICompositeMessageBus compositeMessageBus, IMessageBusTarget messageBusTarget, string name)
    {
        if (name != null && compositeMessageBus != null)
        {
            return compositeMessageBus.GetChildBus(name);
        }
        if (messageBusTarget != null)
        {
            return messageBusTarget.Target as IMasterMessageBus;
        }
        return null;
    }

Steps to reproduce:

Configure a bus (not a child bus) to use the Outbox plugin.
Publish a message so that it is spooled to sql
When OutboxSendingTask attempts to find the 'Main' bus, it will satisfy the condition first condition of the GetBus method and attempt to resolve a ChildBus instead of falling through to messageBusTarget.Target.
GetBus returns null, which aborts the message delivery

Workaround:
Configure SlimMessageBus to add the bus as a child.
builder.Services.AddSlimMessageBus(builder =>
{
    builder.AddChildBus("SVB", bus =>
    {
        ...
    });

    ...
});

Thanks for reporting. I can have a look.


i18n: The same &-shortcut

https://github.com/vim/vim/blob/4a5711b5ea54525a262a57ec0d418f76504bad46/runtime/lang/menu_de_de.latin1.vim#L61
S is already used for "Suchen".

have you contacted the maintainer (mentioned at the top)?

have you contacted the maintainer (mentioned at the top)?

Jón seems not to be on GitHub...

how about sending a mail then?


Setting the seccompProfile in the SecurityContext for containers

It would be great if the seccompProfile in the SecurityContext could be configured for the containers, or set by default. The configuration would probably need to be done here: https://github.com/bank-vaults/vault-secrets-webhook/blob/main/pkg/webhook/pod.go#L858
This is particularly useful for the init containers, where the seccompProfile is set on the main container rather than the pod, and where the seccompProfile configuration is enforced.

@jon-rei I created a PR for this a month ago, but have not gotten any feedback since:
https://github.com/bank-vaults/vault-secrets-webhook/pull/218


ARROW-18121: [Release][CI] Use Ubuntu 22.04 for verifying binaries

APT/Yum verifications use Docker. If we use old libseccomp on host, some operations may cause errors:
https://github.com/ursacomputing/crossbow/actions/runs/3294870946/jobs/5432835953#step:7:5437
+ valac --pkg arrow-glib --pkg posix build.vala
error: Failed to close file descriptor for child process (Operation not permitted)
error: Failed to close file descriptor for child process (Operation not permitted)
error: Failed to close file descriptor for child process (Operation not permitted)
error: Failed to close file descriptor for child process (Operation not permitted)
error: Failed to close file descriptor for child process (Operation not permitted)
error: Failed to close file descriptor for child process (Operation not permitted)

Is this still needed now that https://github.com/apache/arrow/pull/14477 has been merged?
(we can of course use 22.04 nevertheless)

Is this still needed now that #14477 has been merged?

No. We don't need this. #14477 solved the problem.
I merge this as just for using newer Ubuntu.


Improving get_menu_item to handle correctly max_depth.

There can be scenarios where max_depth gets a negative value, e.g. by
calling get_children directly. Currently this leads to the parsing
of the whole tree. Only taking positive values into account provides
the intended behavior.

cherry-picked to develop
thanks @mbehrle


Immediate crash upon startup

What steps will reproduce the problem?
Crashes immediately upon trying to start
How often does this problem occur?
Every time I try to open Grabber
What version of the program are you using? On what operating system?
5.2.4 on Windows 10
Please provide any additional information below
Last version that starts without crashing is 5.2.2

Strange as many others didn't get this issue.
Let's see a few things:

Could you share your main.log file?
Do you even see the "start without restoring the last session" window?
Can you try to backup then remove your restore.igl as well as your tabs.txt files?
Same thing for the settings.ini file.
If that fixes the problem, could you also share those files so I can reproduce (and then push a fix)?

Thanks and sorry for the inconvenience :smile:

screenshot:
log file
main.zip
Deleting the settings.ini file worked. Removing the other two didn't change anything.
Thanks

Without the previous settings.ini file I won't be able to reproduce, but from the fact that the log file is empty when the program crashes, it seems that the initial settings file was either corrupt or locked.


Realigning body on Chrome on Android

When on Chrome on Android, the rules for [data-hint]:after are making html and body narrower than the window:

After deleting the rule:

Oddly enough, fixed by removing 100% height from html.


feathers-authentication with oauth2 fails after first login when using feathers-mongoose

Steps to reproduce

Use the feathers cli to create a new project using the following (default) options:

What type of API are you making?
◉ REST
◉ Realtime via Socket.io

CORS configuration
❯ Enabled for all domains

What database do you primarily want to use?
❯ MongoDB

What authentication providers would you like to support?
◉ google

Then add google api key and secret to the config file.
Run the app with npm start.
Navigate to http://localhost:3030/auth/google
This works fine, you see the default success page with the token
Now navigate to http://localhost:3030/auth/google again
And now you'll get the failure message

This is the output from the debugger:
  feathers-authentication:oauth2 Updating user: [user id would be here] +11m
  feathers-authentication:middleware An authentication error occurred. { GeneralError: Unknown modifier: $__
    at GeneralError.ExtendableBuiltin (/Users/so/Private/_TEST/feathersjs/auth-test/node_modules/feathers-errors/lib/index.js:21:28)
    at GeneralError.FeathersError (/Users/so/Private/_TEST/feathersjs/auth-test/node_modules/feathers-errors/lib/index.js:96:116)
    at new GeneralError (/Users/so/Private/_TEST/feathersjs/auth-test/node_modules/feathers-errors/lib/index.js:281:109)
    at errorHandler (/Users/so/Private/_TEST/feathersjs/auth-test/node_modules/feathers-mongoose/lib/error-handler.js:32:31)
    at process._tickCallback (internal/process/next_tick.js:109:7)
  type: 'FeathersError',
  name: 'GeneralError',
  code: 500,
  className: 'general-error',
  data: undefined,
  errors: {} } +27ms

Expected behavior
The user should be able to login after the first initial login.
Actual behavior
What actually happens is, that the user is created and logged in the first time, but the next times the user is not logged in because of the before mentioned error.
System configuration
Tell us about the applicable parts of your setup.
Module versions (especially the part that's not working):
From package.json:
"dependencies": {
    "body-parser": "^1.17.1",
    "compression": "^1.6.2",
    "cors": "^2.8.3",
    "feathers": "^2.1.1",
    "feathers-authentication": "^0.7.12",
    "feathers-configuration": "^0.3.3",
    "feathers-errors": "^2.6.2",
    "feathers-hooks": "^1.8.1",
    "feathers-mongoose": "^3.6.2",
    "feathers-rest": "^1.7.1",
    "feathers-socketio": "^1.5.2",
    "mongoose": "^4.9.2",
    "passport": "^0.3.2",
    "passport-google-oauth20": "^1.0.0",
    "passport-google-token": "^0.1.2",
    "serve-favicon": "^2.4.2",
    "winston": "^2.3.1"
  }

NodeJS version:
v7.7.4
Operating System:
macOS Sierra 10.12.3
Browser Version:
Chrome 56.0.2924.87

BTW I've tested the same scenario using a clean install from the feathers cli with MariaDB, NeDB and SQLite, using the same google api credentials and it works with all of them, only with feathers-mongoose I get this error.

Can you try the same thing with the new releases of these modules?  You can use the prerelease version of the CLI.  npm install feathers-cli@pre -g  Then feathers generate app and feathers generate authentication.

Ok, so the login now works after the first login. But a few things have changed and/or are missing:

the config file for production is not updated when generating authentication
the settings for google auth changed and now the scope can't be inside of permissions and hostedDomain does not work anymore
and only the googleId is stored in the database, instead of all the user data

I don't know if this is the right place for this feedback, but maybe you can pass it on or point me to the right place.
Oh, and is there a documentation for the new version? I would like to check that out some more.
Thanks

@iamso The latest docs for the new version are here: https://docs.feathersjs.com/v/auk/.  You'll notice that the docs have seen a lot of updates.
In the new version, the authentication has been split into multiple plugins based on authentication strategy.

The core auth plugin is here: https://docs.feathersjs.com/v/auk/api/authentication/server.html
The OAuth2 plugin is here: https://docs.feathersjs.com/v/auk/api/authentication/oauth2.html

The issues you've mentioned would now fit best in this repo: https://github.com/feathersjs/feathers-authentication-oauth2/

@marshallswain Thank you.
But is there any way to solve this in the current version?

Of course there is.  I'm not sure why it's not working in the first place.  I've created a lot of apps with the same setup minus the Google auth.  I've only used GitHub.  Have you tried other providers?

I've tested it with Bitbucket, Facebook, Github, Google and Linkedin, I always get the same error when logging in after the user has been created.

@marshallswain any news?

I'm getting closer to being able to check this.  Have you made any debugging progress?

No, not really, but I haven't spent any more time on it, to be honest.

My hunch is that this has something to do with https://github.com/feathersjs/feathers-authentication-oauth2/pull/20


Understanding Users Accounts and Groups

Hi,
I have just come across this framework and it seems like a great jumping off place for a project i am playing with. However i am not quite understanding the logic behind the user management.
What i am looking to do is to create some ACL groups that users can belong to, only the admin can add/remove the user from that group, users can belong to multiple groups, and groups can have multiple users. I think i can do this using the Admin Groups and assigning the account to the user and the user to the admin group, but that seems like it has an additional necessary step? Would it not make more sense for all the user information to be in one object, Account and for that to be able to be allocated groups, that i can then query when requests hit the restAPI e.t.c?
I'm probably completely misunderstanding the setup here, so please do correct me if i am wrong!
Thanks
Gareth

Thanks for opening an issue. :cake:

I have just come across this framework and it seems like a great jumping off place...

Not to banter about terminology, but it's a boilerplate not a framework. :stuck_out_tongue_winking_eye:

However i am not quite understanding the logic behind the user management.

Although these links are for Drywall and the code is different, the semantics are very similar.

https://github.com/jedireza/drywall/wiki/Users%2C-Roles-%26-Groups
https://github.com/jedireza/drywall/wiki/Admin-%26-Admin-Group-Permissions

I should port those pages over to the wiki here. Hopefully these help for the time being.

I think i can do this using the Admin Groups and assigning the account to the user and the user to the admin group, but that seems like it has an additional necessary step?

Like I mention at the bottom of the the Users, Roles & Groups wiki page, it's may be good to extend the group functionality to accounts. Also keep in mind that we're working with a non-relational db and data is de-normalized.

Your milage may vary. Hopefully this project can inspire a solution that works best for you use case.
Let's keep this issue open as a reminder to port those wiki pages over.
:pray: thanks for getting involved.


Backup functionality

Some sort of prompt for her to find a path to save the backup of the database
(could also necessitate an "import" function that drops the backed up database into the correct folder)

Probably won't have time for this. Maybe should have something in either users or operators manual on how to backup the system
(Go to file folder where system is stored, copy over to flash drive, drop back in the same folder on new machine)


Feature Request: GPT-2

Thanks for this repo! Code descriptions are well written and code is clear!
It would be worth to provide an implementation of OpenAI GPT-2
Thanks.

Yeah that sounds interesting. Will work on that next when we get time.

Yeah that sounds interesting. Will work on that next when we get time.

https://lab-ml.com/labml_nn/transformers/gpt/ it reuses the transformer implementation so wasn't hard to write. Most of the time was spent on debugging because I accidentally was not using decouple weight decay.

https://lab-ml.com/labml_nn/transformers/gpt/ it reuses the transformer implementation so wasn't hard to write. Most of the time was spent on debugging because I accidentally was not using decouple weight decay.

@vpj cool! thank you. 💯

@vpj cool! thank you. 💯


Trouble installing atom beautify

Description
I'm getting the following issue when I try to install beautify. Can anyone help?

<EMAIL_ADDRESS>install /private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents
node-pre-gyp install --fallback-to-build

Unsupported target version: 0.36.8
npm WARN engine<EMAIL_ADDRESS>wanted: {"node":">= 4.2.0"} (current: {"node":"0.10.40","npm":"2.13.3"})
npm WARN engine<EMAIL_ADDRESS>wanted: {"node":">0.10 <=5"} (current: {"node":"0.10.40","npm":"2.13.3"})
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN engine<EMAIL_ADDRESS>wanted: {"node":">=0.12"} (current: {"node":"0.10.40","npm":"2.13.3"})
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN engine<EMAIL_ADDRESS>wanted: {"node":">=0.12"} (current: {"node":"0.10.40","npm":"2.13.3"})
npm WARN engine<EMAIL_ADDRESS>wanted: {"node":">=0.12"} (current: {"node":"0.10.40","npm":"2.13.3"})
node-pre-gyp ERR! install error
node-pre-gyp ERR! stack Error: Unsupported target version: 0.36.8
node-pre-gyp ERR! stack     at get_runtime_abi (/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents/node_modules/node-pre-gyp/lib/util/versioning.js:156:23)
node-pre-gyp ERR! stack     at Object.module.exports.evaluate (/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents/node_modules/node-pre-gyp/lib/util/versioning.js:277:19)
node-pre-gyp ERR! stack     at install (/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents/node_modules/node-pre-gyp/lib/install.js:138:31)
node-pre-gyp ERR! stack     at Object.self.commands.(anonymous function) [as install] (/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents/node_modules/node-pre-gyp/lib/node-pre-gyp.js:48:37)
node-pre-gyp ERR! stack     at run (/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents/node_modules/node-pre-gyp/bin/node-pre-gyp:79:30)
node-pre-gyp ERR! stack     at Object. (/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/node_modules/atom-beautify/node_modules/remark/node_modules/chokidar/node_modules/fsevents/node_modules/node-pre-gyp/bin/node-pre-gyp:131:1)
node-pre-gyp ERR! stack     at Module._compile (module.js:456:26)
node-pre-gyp ERR! stack     at Object.Module._extensions..js (module.js:474:10)
node-pre-gyp ERR! stack     at Module.load (module.js:356:32)
node-pre-gyp ERR! stack     at Function.Module._load (module.js:312:12)
node-pre-gyp ERR!  Darwin 14.5.0
npm ERR! argv "/Applications/Atom.app/Contents/Resources/app/apm/bin/node" "/Applications/Atom.app/Contents/Resources/app/apm/node_modules/npm/bin/npm-cli.js" "--globalconfig" "/Users/wagnercolodette/.atom/.apm/.apmrc" "--userconfig" "/Users/wagnercolodette/.atom/.apmrc" "install" "/private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/d-11666-897-xjj2sy/package.tgz" "--target=0.36.8" "--arch=x64"
npm ERR! node v0.10.40
npm ERR! npm  v2.13.3
npm ERR! fetch failed with status code 502
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     /private/var/folders/vj/r0zdmg_n1_v5wjy5vv9q105c0000gn/T/apm-install-dir-11666-897-js3ruj/npm-debug.log
Checklist

[ ] I have tried uninstalling and reinstalling Atom to ensure it installed properly
[ ] I have reloaded (or restarted) Atom to ensure it is not a caching issue
[ ] Searched for existing Atom Beautify Issues at https://github.com/Glavin001/atom-beautify/issues
so I know this is not a duplicate issue

Usually this issue is due to missing dependencies.  Try running this from the command line and let us know if it helps:
apm install atom-beautify

I tried doing that, and it reports the same error after about 4 minutes.

Fixed it. Removed all node and node_module directories and uninstalled node, then reinstalled it.


Add .phony target "clean" to Makefile

This CL adds the .phony target "clean" to remove binaries generated during compile.

All tests pass ( "make tests")

Note:
**- I am seeking feedback on additional tests to add or modify

I am seeking feedback on unit test coverage**

NOTE: Community submissions can take up to 2 weeks to be reviewed.
Please take the following actions before submitting this pull request.

Fork your PR branch from the Toolkit "develop" branch (not main)
Test all changes with pre-commit in a local branch #
Confirm that "make tests" passes all tests
Add or modify unit tests to cover code changes
Ensure that unit test coverage remains above 80%
Update all applicable documentation
Follow Cluster Toolkit Contribution guidelines #

Accidental request to merge to "main"


[BUG] - google.api_core.exceptions.MethodNotImplemented: 501 Received http2 header with status: 404

What happened:
I ran the datacatalog-util in GCP Cloud shell and it throw exception below:
It is throwing below error while running on GCP Cloud Shell.
Could you please help me to resolve it.
Error Log=============
status = StatusCode.UNIMPLEMENTED
details = "Received http2 header with status: 404"
debug_error_string = "{"created":"@1616126950.574176093","description":"Received http2 :status header with non-200 OK status","file":"src/core/ext/filters/http/client/http_client_filter.cc","file_line":129,"grpc_message":"Received http2 header with sta
tus: 404","grpc_status":12,"value":"404"}"
What you expected to happen:
Ideally it should start updating the Dataset table in Data Catalog based in the .csv file data
How to reproduce it (as minimally and precisely as possible):
Create GCP project and service account with needed permissions.
Run as the guide document described
Anything else we need to know?:

Hi @sandeep2sky, thanks for opening this!
I'm here to help, could you provide me more information about the issue?

What command did you try to execute?
Was it the datacatalog-util tags create --csv-file CSV_FILE_PATH command?

Did you follow the sample-input, like: tags-opt-1-all-metadata.csv?

From the error message, I'd guess that the Data Catalog client library is not able to find the entry to update the tags, this might be something invalid in the CSV file, like the linked_resource.... Do you also have created the tag template before?
Some context: I just ran the create tags tutorial with a new table at:
create-the-tags, and was able to successfully execute it.
Are you using the default Big Query location or some specific region in your table/dataset? To help me troubleshoot this.

This is linked to https://github.com/ricardolsmendes/datacatalog-tag-manager/issues/39

@ricardolsmendes to be aware.

Thanks @mesmacosta @ricardolsmendes. The issue got resolved. It was due to csv file. I was using csv file that I have saved  from an Excel file in the predefined format. I tried to use the CSV from the GitHub provided in the package itself and it resolved the issue. Layout and content was same but it works with csv from repo. Thanks for your support and comment.

thanks @sandeep2sky !


BIND module fails for empty db_records_folder parameter

Expected behaviour

User instantiates bind-server and provides empty db_records_folder
The internal null_resource detects the empty parameter and skips copying any folder

Actual behaviour

"${var.db_records_folder}/" is interpolated into /
Calling user sends / to /tmp/db_records
Permission failure to access all files under / fails the provisioner
Local / files are available on the instance
terraform apply fails

Proposed solution
Send a dummy folder that gets removed upon remote_exec.

I believe these changes have been merged, though we haven't seen them run recently. I am going to close the issue as complete and open new tickets in the future if there are still issues here.


Increase number of process workers for assessment-store

Assessment-store can sometimes receive multiple API requests for the same high-level request originating from assessment. It would be beneficial to have multiple workers to be able to process concurrent requests.
Change description
A brief description of the pull request

[ ] Unit tests and other appropriate tests added or updated
[ ] README and other documentation has been updated / added (if needed)
[ ] Commit messages are meaningful and follow good commit message guidelines (e.g. "FS-XXXX: Add margin to nav items preventing overlapping of logo")

How to test
If manual testing is needed, give suggested testing steps
Screenshots of UI changes (if applicable)

I think this will be an improvement, but it should not be actually one request per worker i.e. as described in https://stackoverflow.com/a/48543832 and https://github.com/communitiesuk/funding-service-design-utils/blob/024fa53fe1be7f5d6a6ccf0f35f90febc1c9c69c/fsd_utils/gunicorn/config/devtest.py#L76C1-L76C19

I think this will be an improvement, but it should not be actually one request per worker i.e. as described in https://stackoverflow.com/a/48543832 and https://github.com/communitiesuk/funding-service-design-utils/blob/024fa53fe1be7f5d6a6ccf0f35f90febc1c9c69c/fsd_utils/gunicorn/config/devtest.py#L76C1-L76C19

Isn't that only applicable if we're using gevent / greenlet workers?