Commit Graph

8 Commits

Author SHA1 Message Date
Sean Hatfield
0074ededdd
Github data connector improvements (#2439)
* fix tree/blob github urls from branches not being loaded

* improve ux of github data connector

* lint

* patch Github URL parser to just validate with `URL` native parser

* uncheck LocalStorage of PAT for security reasons

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-10-21 15:25:35 -07:00
Timothy Carambat
30645831a1
1959 filetype filters (#2378)
* Updated the `GitHubRepoLoader` class to use the new import syntax and adjust the `recursiveLoader` method accordingly.

* add @langchain/community to collector package.json

* fix: Improve handling of complex ignore patterns in GitLabRepoLoader

* refactor: use ignore package for simplified ignore logic

* run yarn lint

* add @langchain/community@^0.2.23

* remove unused dep
lint

---------

Co-authored-by: Emil Rofors (aider) <emirof@gmail.com>
2024-09-26 12:50:35 -07:00
Blazej Owczarczyk
b2123b13b0
Added an option to fetch issues from gitlab. Made the file fetching a… (#2335)
* Added an option to fetch issues from gitlab. Made the file fetching asynchornous to improve performance. #2334

* Fixed a typo in loadGitlabRepo.

* Convert issues to markdown.

* Fixed an issue with time estimate field names in issueToMarkdown.

* handle rate limits more gracefully + update checkbox to toggle switch

* lint

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-09-26 11:45:18 -07:00
Timothy Carambat
4fa3d6d333
Load all branches in gitlab data connector (#2325)
* Fix gitlab data connector for self-hosted instances (#2315)

* Linting fix.

* Load all branches in the GitLab data connector #2319

* #2319 lint fixes.

* update fetch on fail

---------

Co-authored-by: Błażej Owczarczyk <blazeyy@gmail.com>
2024-09-19 13:34:38 -05:00
Blazej Owczarczyk
b25298c04a
Fix gitlab data connector for self-hosted instances (#2315) (#2316)
* Fix gitlab data connector for self-hosted instances (#2315)

* Linting fix.
2024-09-18 16:12:15 -05:00
timothycarambat
9aa77dfb8d Add verbose logging to GH loader
connect #2243
2024-09-09 14:36:37 -07:00
Mehmet Ünlü
0d4560b9e4
2049 remove break that prevents fetching files from gitlab repo (#2050)
fix: remove unnecessary break

Remove unnecessary break that prevents checking next pages for blob objects.
2024-08-06 10:17:55 -07:00
Timothy Carambat
42235fcd8a
GitLab Hosted and Local Connector (#1932)
* Add support for GitLab repo collection as well as Github Repo collection
* Refactor for repo collectors to be more compact

---------

Co-authored-by: Emil Rofors <emirof@gmail.com>
2024-07-23 12:23:51 -07:00